Ai2's newly released Olmo-core 3 open training framework scales sparse model training toward a trillion parameters while documenting the measurement traps that make performance claims hard to judge.
A new BioNeMo recipe from NVIDIA combines grouped expert kernels, fused MXFP8 8-bit math and kernel fusion to train mixture-of-experts biological foundation models. In NVIDIA's own benchmark on eight B200 GPUs, it reached up to 2.21x the throughput of the Hugging Face baseline. Here is what was measured, what was not, and why it matters for labs without hyperscale budgets.