The reporting

2 articles

Topic: quantization

Screenshot of the llama.cpp web interface running a local small language model, illustrating local small-model inference in an agent harness context

ANALYSIS AI Research

Sixteen small models, sixteen failures: the preprint puncturing the 'good enough' assumption inside AI agents

A new preprint systematically tested whether small language models can handle the tiny decisions surrounding an AI agent's main planner. Across 16 configurations at best settings, none passed the paper's pre-specified quality bar, and 4-bit quantization changed nothing.

Close-up photograph of the side of a 2021 16-inch MacBook Pro showing the Touch ID button and SD card slot, representative of the Apple Silicon laptops targeted by the new GGUF support.

EDUCATIONAL AI Applications

Quantized AI models just got a second front door: transformers now runs GGUF checkpoints

Hugging Face's transformers library can now load GGUF quantized checkpoints, the compressed files that power Ollama and LM Studio. Here is what quantization is, why the September 22, 2026 announcement matters for running AI on your own machine, and where the caveats are.