NVIDIA publishes an open recipe that trains biological AI models at up to 2.21x baseline throughput

NVIDIA published a step-by-step tutorial on September 24, 2026 showing how to train a class of large AI models used in genomics and protein design far more cheaply than a common open-source baseline. The technique, released through the company's BioNeMo recipes repository, combines three changes at once: batching the work of many expert subnetworks into single GPU operations, switching key calculations from 16-bit to block-scaled 8-bit arithmetic, and fusing several of those steps into one kernel. In the company's own benchmark, the resulting recipe trained a Mixtral-8x7B configuration at up to 2.21 times the throughput of the Hugging Face baseline on eight NVIDIA B200 GPUs.

The result is a vendor-reported number on a single configuration, not an independent evaluation, and it says nothing by itself about biological accuracy. But it points at a real bottleneck. Models that read DNA and design proteins keep getting bigger, and the electricity and GPU-hours needed to train them scale with size. If the throughput gain generalizes even partially, a training run that once required hyperscale budgets moves closer to what a university lab or a modestly funded biotech startup can afford. That is the practical stake of what would otherwise look like a dry systems-engineering tutorial.

What mixture-of-experts means, without the jargon

Most large AI models today are dense transformers: every token of input, every piece of sequence, passes through every part of the network on its way through the model. That design is simple and predictable, but it means capacity is expensive. If you want the model to know more, you make every layer bigger, and every training step costs proportionally more compute.

Mixture-of-experts architectures take a different route. Instead of one large feed-forward block, the model contains many smaller specialist blocks, called experts. A router looks at each token and sends it to only a few experts, perhaps two out of eight. The total model can therefore hold far more parameters, and far more potential knowledge, while each token only pays the compute cost of the handful of experts that actually process it. Models in this family include Mixtral 8x7B, the configuration NVIDIA used for its benchmark.

The biological connection is direct. NVIDIA's BioNeMo platform exists to train foundation models that work on protein sequences, genomes and molecular data, and those workloads increasingly borrow architectures from the language-modeling world. Long genomic sequences add another pressure: the model must hold far more intermediate data in memory during training than a short text prompt would require.

Three bottlenecks, three fixes

The recipe addresses three distinct inefficiencies, each documented in the tutorial with code.

First, fragmented expert kernels. The tutorial shows that the Hugging Face implementation of Mixtral iterates over all experts in a Python loop, so each expert triggers its own separate GPU kernel launch. On a GPU, launching an operation has real overhead, and launching many small operations one at a time leaves the hardware underused. NVIDIA's Transformer Engine library instead provides a GroupedLinear primitive that gathers the expert weights and submits all of their matrix multiplications as one grouped operation. The tutorial's figure contrast is blunt: one loop of many calls versus one grouped call.

Second, memory. Model weights and activations are normally held in 16-bit BF16 format during training. The recipe instead uses MXFP8, an 8-bit format with block scaling, which assigns a scaling factor to each block of 32 consecutive values. That block-level scaling helps preserve numerical range and accuracy compared with coarser scaling schemes, and on NVIDIA's Blackwell GPUs the format is hardware-accelerated in the Tensor Cores. Cutting the bit width roughly halves the memory footprint of weights and activations, which matters most for genomics workloads with long sequences.

Third, quantization overhead. In low-precision training the master weights stay in 16 bits, so the framework must convert values to 8 bits before the fast math and back afterward. Done naively, those conversions are separate operations that add overhead of their own. The recipe uses the Transformer Engine Sequential API to recognize the pattern of grouped linear layers, the SwiGLU activation and routing-weight scaling, and replace the whole sequence with a single fused kernel, ForwardGroupedMLP_CuTeGEMMSwiGLU_MXFP8, plus a matching fused backward operation. Intermediate results that would otherwise be written to memory and read back are never materialized.

What was measured, in NVIDIA's own words

The tutorial's benchmark section is explicit about what was measured.

NVIDIA's exact wording, from the Results section of the tutorial:

In our training benchmark on eight NVIDIA B200 Tensor Core GPUs, the recipe delivered up to 2.21x the throughput of the Hugging Face baseline.

Attribution: Faradawn Yang, Peter St John, Kyle Tretina and Zoey Zhang, authors of the NVIDIA Technical Blog tutorial "Efficient MoE Training for Biological Foundation Models", published September 24, 2026.

A figure in the post labels the measurement as Mixtral-8x7B training throughput on eight B200 GPUs against the Hugging Face baseline. The tutorial also names a baseline starting point for reproducibility: a two-GPU sanity configuration to confirm expert parallelism works before scaling to the full Mixtral-8x7B run with expert parallelism across eight GPUs.

Several qualifications are necessary and the authors themselves supply most of them. The 2.21x figure is NVIDIA's own measurement of its own optimization against a baseline that the tutorial describes as running experts one at a time in a Python loop, which is close to a worst-case implementation. The phrase "up to 2.21x" implies the gain varies, likely by configuration, sequence length and precision settings; the tutorial does not publish a table of per-configuration numbers in the retrieved text. The benchmark measures training throughput, tokens processed per unit time, not model quality, and no biological accuracy results are presented. And it was run on NVIDIA's newest hardware, with the fused 8-bit kernel requiring Blackwell-class GPUs, so labs on older equipment would see smaller or different gains.

What it means for labs without hyperscale budgets

Throughput is a means, not an end. A faster trainer only matters to biology if the models it produces are as good as or better than what labs could otherwise afford. The tutorial does not claim accuracy results, and this article does not find any published third-party replication of the benchmark. That gap is the main reason to treat the headline number as encouraging rather than settled.

Still, the direction of the work is significant for access. Two structural costs dominate the budgets of groups training biological foundation models: GPU-hours and GPU memory. The recipe attacks both at once. If the grouped-kernel and 8-bit techniques deliver even half of the reported gain on other MoE configurations, the minimum viable budget for training a competitive biological model shifts downward, which widens the set of institutions able to build rather than merely consume these models. That aligns with a broader pattern in open AI infrastructure: efficiency work, once published as recipes and code, tends to be adopted quickly by groups that were previously priced out.

The recipe is open source in NVIDIA's BioNeMo recipes repository, so the techniques are inspectable and reproducible in principle. Independent benchmarks on non-NVIDIA hardware, or comparisons against stronger baselines than the Python-loop implementation, would be the natural next evidence. Until then, the fairest reading is: NVIDIA has published a plausible, well-engineered efficiency recipe and reported a large gain on one benchmark of its own choosing.

The fine print: whose hardware, whose benchmark

The benchmark hardware requirement deserves emphasis. The fused MXFP8 grouped kernel runs only on NVIDIA Blackwell GPUs, the company's current data center generation. A lab with an older fleet of A100 or H100 GPUs can still adopt the grouped expert execution and fusion ideas, but the headline 8-bit path, and much of the memory and speed advantage described here, is gated on buying into NVIDIA's latest silicon. The tutorial is, among other things, an argument for that upgrade.

That is not a criticism of the engineering, which is documented in unusual detail, down to the code snippets and the launch commands. But readers should recognize the shape of the document: it is a vendor tutorial whose authors are NVIDIA employees, published on NVIDIA's own blog, benchmarked on NVIDIA's own hardware, promoting a recipe in NVIDIA's own repository. The technical content appears sound and the claims are carefully scoped. The number to remember is not 2.21x in the abstract; it is 2.21x, on one Mixtral-8x7B configuration, on eight B200s, against the Hugging Face baseline, as reported by NVIDIA and not yet independently confirmed.