The reporting

3 articles

Topic: ai benchmarks

A hand-drawn analog circuit schematic showing a grounding resistance tester, illustrating the schematic-to-netlist task at the heart of the SGAnalog benchmark

ANALYSIS AI Research

Grading AI Against Real Silicon: The SGAnalog Benchmark Scores Models on 273 Circuits That Actually Reached a Chip Factory

A new benchmark from two Berkeley researchers grades AI models on schematic transcription and device sizing using 273 topologically distinct open-source circuits from Tiny Tapeout manufacturing shuttles. The results reveal sharp limits in visual reasoning, and a refusal problem that has nothing to do with skill.

An over-the-shoulder view of a person writing code on a laptop, illustrating software agents that operate computers and code environments

NEWS AI Models

H Company's Holo4 puts an open-weights computer operator on Hugging Face: one model that clicks, codes, and calls APIs, with every step it took published

H Company released Holo4 as open weights in 27B dense and 35B-A3B Mixture of Experts sizes on September 28, 2026, scoring 61.7 percent on OSWorld 2.0 for the 27B model against 81.8 percent for Opus 5.5, publishing 7,366 agent trajectories for public inspection, and shipping a smaller Holotron4 Nano. Cost comparisons are self-reported, and the weights carry a non-commercial license.

A 3D-rendered sketch of an NVIDIA GPU workstation, representing the GPU clusters used to train large mixture-of-experts language models.

ANALYSIS AI Infrastructure

Ai2 opens its trillion-parameter training playbook with Olmo-core 3

Ai2's newly released Olmo-core 3 open training framework scales sparse model training toward a trillion parameters while documenting the measurement traps that make performance claims hard to judge.