The reporting

2 articles

Topic: mixture-of-experts

A 3D-rendered sketch of an NVIDIA GPU workstation, representing the GPU clusters used to train large mixture-of-experts language models.

ANALYSIS AI Infrastructure

Ai2 opens its trillion-parameter training playbook with Olmo-core 3

Ai2's newly released Olmo-core 3 open training framework scales sparse model training toward a trillion parameters while documenting the measurement traps that make performance claims hard to judge.

A molecular model of a DNA double helix, illustrating the biological foundation models that NVIDIA's new training recipe targets.

NEWS AI Infrastructure

NVIDIA's open recipe trains mixture-of-experts biological AI models at up to 2.21x baseline throughput

A new BioNeMo recipe from NVIDIA combines grouped expert kernels, fused MXFP8 8-bit math and kernel fusion to train mixture-of-experts biological foundation models. In NVIDIA's own benchmark on eight B200 GPUs, it reached up to 2.21x the throughput of the Hugging Face baseline. Here is what was measured, what was not, and why it matters for labs without hyperscale budgets.