OpenAI's GPT-6 caching stores more of what an AI already read, cutting repeated input costs by up to 90% and trimming wait times

The AI that reads everything again, every turn

Every time you send a new message to an AI assistant, the system behind it typically rereads the entire conversation from the beginning. The instructions the developer wrote, the tool definitions, the documents you pasted in, and every earlier turn all have to be processed again before the model can answer your latest question. For short chats that cost is invisible. For long-running AI agents that work for hours, such as coding assistants or research agents, it is a large and growing share of both the price you pay and the time you wait.

On September 22, 2026, OpenAI announced an improved prompt caching system for its GPT-6 family of models, alongside new developer tools for measuring and diagnosing how well caching works. The announcement, published on OpenAI's news site, states that cached input tokens reused within a 30-minute window are discounted by up to 90 percent, and that the GPT-6 family ships with higher cache hit rates by default. This article explains what prompt caching is, what OpenAI says changed, and which claims come from the vendor rather than from independent testing.

What prompt caching actually stores

When a language model processes input, it computes internal states, known in the industry as key-value states, that let the model refer back to earlier tokens while it works. OpenAI's developer documentation explains that prompt caching preserves those states for a reusable prefix: the unchanged tokens at the beginning of a prompt. When a later request starts with the same prefix and finds a matching stored entry, the model reuses the saved work instead of recomputing it. It still processes any genuinely new input.

The documentation describes three benefits in plain terms. It is compute-efficient, because the model avoids recalculating a prefix it has already processed. It is cheaper, because reused tokens are billed at a reduced cached-input rate, discounted up to 90 percent. And it is faster, because less time is spent processing input before the first word of the response appears. Caching is enabled by default on supported models, and a prompt must meet a minimum cacheable length, stated as 1,024 tokens for GPT-5.6 and later models, before anything can be cached.

One detail from the documentation matters for understanding the economics: the discount applies to reads from the cache, while the first request that writes a prefix to the cache costs 1.25 times the standard input rate. OpenAI's guide gives a worked example: writing a prefix once and reusing it once costs 1.35 times its ordinary input cost, compared with 2 times for processing it twice without caching. The savings grow with each additional reuse. Across ten requests, one cache write and nine full cache reads cost 2.15 times, versus 10 times without caching.

What OpenAI announced on September 22, 2026

The caching improvement was announced the same day OpenAI introduced GPT-6 Sol and GPT-6 Luna, two models that expand the GPT-6 family launched earlier in the month with GPT-6 Astra. The Sol and Luna announcement states that improvements in caching and inference let OpenAI serve these models at lower cost, and that API prices for Sol and Luna are reduced by 50 percent compared with their GPT-5.6 promotional pricing.

The caching announcement itself describes the headline terms. OpenAI now gives cache discounts for eligible shared prefixes reused within a 30-minute window, and the company states that discounts reach up to 90 percent on cached input tokens. According to the developer guide, the 30-minute lifetime is the default and, for GPT-5.6 and later models, the only supported value that can be set directly, though OpenAI may retain entries longer. Reusing a prefix refreshes its lifetime without another cache-write charge. These are OpenAI's stated terms, not independently audited figures.

It is worth separating what changed here. This is not a new model and not a new capability. It is infrastructure work that changes how much of the reading a system already did gets reused, which is exactly the kind of change that shapes what a long AI session costs in practice.

New tools for seeing whether caching is working

The announcement is as much about tooling as about the discount itself, because caching only saves money if it actually engages, and it can silently stop working when a prompt changes.

OpenAI introduced a Prompt Caching Dashboard that shows how much of an application's input is served from cache, tracks hit rates over time, and compares cached and uncached tokens in an input composition chart. When an unexpected cache miss occurs, a diagnostics tool compares a request against a recent response to identify what changed, such as the model, tools, settings, or input, that prevented reuse. The announcement includes an example diagnostic output showing a cache miss caused by changed tools, with 5,629 reusable tokens affected.

Developers also gained more control over where caching happens. Explicit cache breakpoints let developers choose which prompt prefixes get written to the cache, so stable context can be cached while frequently changing content stays at the end of the prompt at the normal input rate. Each request can create up to four cache writes. On GPT-6 models, developers can now change the reasoning effort between responses, raising it for a hard task or lowering it for a routine follow-up, without breaking the cache, by using a configuration update. Similarly, tool definitions can be kept in place while individual tools are enabled or disabled, preserving the earlier context for reuse. A prewarming option lets an application prepare known context ahead of time, such as during startup, so the model starts responding sooner when the first user request arrives.

What customers report, on OpenAI's say-so

The announcement includes customer testimonials from GitHub, Wordsmith, Manus, and Strawberry Browser. These are vendor-supplied statements published by OpenAI, not independently verified measurements, and readers should treat them as such. With that caveat, they give a sense of the reported magnitude.

Mario Rodriguez, Chief Product Officer at GitHub, is quoted saying that OpenAI's prompt caching plays a critical role in GitHub Copilot's performance and that, over the past several months, GitHub reduced by more than 50 percent the share of prompt tokens requiring fresh processing across billions of requests to OpenAI models, relative to its previous baseline.

Manus, which builds long-running agents, is quoted through Bin Fan, Agent Team Lead, saying that after refining cache breakpoint placement with OpenAI's engineering team, its OpenAI model cache hit rate went from roughly 85 percent to consistently above 90 percent in less than a week, further lowering inference costs in production.

Eugene Mikhantyev, AI Engineer at Strawberry Browser, is quoted saying that after moving session agents to explicit cache breakpoints, cache hit rates on its evaluations rose from 83 percent to 91 percent in under a week, that cache writes fell by roughly two-thirds, and that inference costs fell by 36 percent.

A Wordsmith executive, Arian Hanifi, Chief Technology Officer, is quoted saying the diagnostics and dashboard helped improve cache hit rates by a few percentage points, reducing costs by 20 percent, and that explicit breakpoints made it economically viable to fork conversations for background tasks while reusing nearly all shared context.

A few of these figures can be sanity-checked against OpenAI's own pricing mechanics. A cache hit rate rising from 83 percent to 91 percent means roughly 8 percent more of the input is billed at the 0.1x cached rate rather than the standard rate, so a 36 percent total inference cost reduction also depends on other factors such as fewer cache writes and output pricing. The numbers are plausible but specific to each company's workload, and none have been independently audited.

What this means for readers and builders

For most everyday users, none of this appears as a settings menu. Its effects show up indirectly. Agents that maintain long context, such as coding assistants working in a large codebase or research agents that have ingested many documents, should respond faster on follow-up requests and cost their operators less, savings that may or may not be passed on depending on the product. OpenAI's own reduction of Sol and Luna API prices by 50 percent, attributed in part to caching and inference improvements, is a direct example of that chain.

The 30-minute window is the practical limit to know about. A cached prefix stays eligible for reuse for 30 minutes after its most recent write or reuse. A user who returns to a conversation hours later should expect the system to reread the earlier context at full price, which can mean a slower and more expensive first response after a long gap. Caches are also stored on individual machines and are not shared across organizations, per the documentation, so traffic routing affects whether a matching entry is found.

For developers, the announcement's most useful contribution may be observability. Before this, a cache miss was often invisible; now a dashboard and a diagnostics tool can show what changed and roughly how many tokens were affected, which turns caching from a background hope into something a team can measure and manage.

In our view, this announcement is more consequential than its modest framing suggests, because the economics of persistent agents have been the main barrier to longer, more capable AI sessions. But the headline numbers, including the 90 percent discount, the 30-minute window, and every customer quote, are OpenAI's own statements. Independent benchmarking of real-world cache hit rates across providers does not yet exist, and we will flag it if it appears.