The reporting

2 articles

Topic: latency

NVIDIA CEO Jensen Huang holds a desktop-class RTX Blackwell graphics card at the company's CES 2025 keynote in Las Vegas.

ANALYSIS Infrastructure

Green Contexts Go Mainstream: CUDA 13.1 Lets One App Carve a Guaranteed GPU Slice for Urgent Work

CUDA 13.1 brings green contexts to the mainstream Runtime API, letting an application reserve a fixed slice of streaming multiprocessors so a small, latency-critical job never waits behind a bulk workload. NVIDIA reports 0.007 ms versus 0.140 ms on a Blackwell GPU; all figures are vendor-reported.

Closeup photograph of the rear of a server rack at the NERSC data center, showing blue LED screens and network cables on dense compute hardware.

NEWS AI Infrastructure

OpenAI's GPT-6 caching stores more of what an AI already read, cutting repeated input costs by up to 90% and trimming wait times

OpenAI's September 22, 2026 announcement details a rebuilt caching system for GPT-6 that stores more of the context an agent already processed, discounts reused input tokens by up to 90 percent within a 30-minute window, and gives developers new dashboards for diagnosing cache misses. Here is what that means for the cost and speed of long AI sessions.