LLM Prompt Caching Cost Savings Across Providers

Most production LLM apps send the same large chunk of text with every single request: system instructions, tool definitions, documents pulled from a database, the back-and-forth history of a conversation. When a new request shares a prefix with one already processed, the provider pulls from a stored cache instead of recomputing it, and charges a fraction of the normal rate for those tokens. Anthropic, OpenAI, Google Gemini, and DeepSeek all offer some version of this, but the discount size, the setup required, and the rules for what counts as a "match" differ enough between them that assuming caching just works, the same way, everywhere, is how teams end up leaving real money on the table.

How prefix structure controls KV caching

To understand why these provider differences matter, it helps to know what's actually being cached. When a model processes a prompt, it builds key and value vectors for every token at every layer of the transformer. Matching only works on a prefix: a contiguous block of text starting at position zero. The match also has to be exact. Changing even one character before the cache boundary breaks the cached prefix from that point forward, though some systems can still recover a shorter matching sub-prefix through backward lookback. Anything that comes after the cache boundary, like the user's actual question or fresh search results, gets computed from scratch no matter what. The discount only touches the stable part of the prompt. Prompt order matters so much because static material (tool definitions, system instructions, long reference documents) needs to sit at the top, and anything that changes request to request needs to go last. Every provider also sets a minimum prefix length before caching kicks in. Anthropic requires at least 1,024 tokens for Claude Haiku and 2,048 for Sonnet and Opus-class models. None of these caches last forever, either. Every provider runs its cache on a time-to-live window, so a workload with long gaps between requests will keep hitting cold starts and paying full price, no matter how well the prompt is structured.

Anthropic's explicit caching model: the deepest discount with the highest implementation control

Anthropic offers the deepest discount of any major provider, and it asks the most from developers in return. Caching here isn't automatic: it has to be turned on by adding a cache_control field to the request, with up to four breakpoints allowed in a single call. That means a team can cache its tool definitions, its system prompt, and its RAG context as three separate, independently managed blocks. Cached tokens that get read back cost a small fraction of the standard input rate, a substantial discount, which pays off the added setup work. Writing to the cache costs more than a normal request, though: somewhat more than standard for a 5-minute cache, or roughly double for a 1-hour cache. A 5-minute cache write pays for itself after just one cache read. A 1-hour cache write pays for itself after two reads. Passing the TTL explicitly on every request, rather than trusting the default, avoids that trap. Anthropic's caching also stacks with its Batch API, which cuts token costs in half on its own. Combining batch processing with cache reads brings the compounded discount on the repeated part of a prompt to as much as 95%, the single best deal available anywhere in this comparison. Understanding where those caching dollars actually land, and where they slip away through cache misses or a poorly structured prefix, separates a plan that looks good on paper from savings a team can actually bank. An LLM gateway like Concentrate AI, which tracks spend by model and provider in real time, gives teams running multiple LLM providers a way to measure cache hit rates and realized savings across their whole inference stack, instead of reconstructing the picture from separate provider invoices after the fact.

OpenAI's automatic caching: zero configuration, lower discount ceiling

OpenAI takes the opposite approach from Anthropic: no setup, no markers, no decisions to make. Since October 2024, every API call carrying 1,024 or more input tokens is automatically eligible for caching, with nothing required from the developer. That convenience comes with a shallower ceiling. The discount varies by model family: roughly 75% off on the GPT-4.1 family, up to 90% on newer model lineups, and close to a full discount on realtime audio models. Cache lifetime under normal conditions runs 5 to 10 minutes, though it can stretch to an hour during off-peak periods under standard in-memory retention, or up to 24 hours with an explicit extended retention setting. There's no write-cost premium either. The first request that populates the cache is billed at the standard rate, with no extra charge for writing it, so a cache miss on OpenAI costs nothing beyond what an uncached request would cost anyway, unlike Anthropic's write multiplier. The tradeoff that comes with full automation is control: because OpenAI decides what gets cached, a team can't explicitly flag a 15,000-token document block the way it can with Anthropic's cache_control markers. Good prompt structure still matters, but there's no dial to turn when something isn't caching the way it should. Because OpenAI's caching runs automatically with no prompt instrumentation, adoption friction is low, but the smaller discount and the 128-token matching granularity mean teams need to actually see how often their requests hit the cache, and whether the savings justify the overhead of keeping long static prefixes around. That's where tracking spend by provider becomes practically useful: it lets a team measure how well OpenAI's caching is performing on its own, and set that result side by side with how the same workload performs on a different provider.

Google Gemini's dual-mode caching: implicit defaults and explicit long-context control

Gemini runs two caching systems at once, and that combination makes it a distinct option rather than a middle ground between Anthropic and OpenAI. Implicit caching is turned on by default on eligible models, no code changes needed, much like OpenAI's automatic approach, and it delivers a 90% discount on cached tokens for Gemini 2.5 and newer models. Alongside that sits an explicit context caching API, built around named cache objects through the cachedContents endpoint, with a configurable TTL that defaults to one hour. This explicit mode is built for long-lived reference material reused across many separate sessions, not just within one ongoing conversation. Gemini's real edge is context size: up to 2 million tokens, so the absolute savings on one large cached document can dwarf anything achievable on Anthropic or OpenAI with a comparable dataset. Cache scope on Gemini is also set per-project rather than per-organization, giving finer isolation than OpenAI's org-wide scope. The implicit layer carries no write-cost premium at all, and the explicit API bills storage by the hour for the initial write.

DeepSeek's disk-backed caching: no write cost, finer granularity, different reliability profile

DeepSeek takes a structurally different route than the other three. Its cache runs on disk rather than GPU memory, and the economics follow from that choice. There's no separate charge for writing to the cache: the first request that fills it costs the same as a standard cache miss, which removes a risk that exists squarely on Anthropic's 1-hour tier, where an unused write can end up costing more than it saves. Matching happens in blocks of 64 tokens, far finer than the 1,024-token blocks Anthropic and OpenAI use. DeepSeek's caching runs automatically, with no explicit markers needed, putting it in the same zero-config category as OpenAI. The no-write-cost model combined with fine-grained matching makes DeepSeek a strong fit for workloads with a lot of prompt variability, or moderate request volume, where a full-prefix match is unlikely. Those are exactly the situations where Anthropic's write premium would cancel out any benefit from caching. DeepSeek isn't automatically the cheapest option across the board. It fits a specific shape of workload well: one where prompts vary enough that other providers would mostly record misses.

How to map workload characteristics to provider caching mechanics

Choosing a caching provider comes down to four things: how stable the prefix is, how often requests come in, how long the prefix runs, and how much write-cost risk a team can tolerate. A high-volume workload with a long, stable prefix and steady traffic is the textbook case for Anthropic's 1-hour TTL tier: the 90% discount is the best available, and with that much volume, the write premium amortizes away almost immediately. A team with no time to instrument prompts, or one just getting started with caching at all, gets immediate value from OpenAI's automatic model: a 50% discount with zero code changes, and no penalty for cache misses since there's no write premium to worry about. A workload built around a very large, stable reference corpus, a regulatory archive, a full codebase, a long research collection, is the structural fit for Gemini's explicit context caching API, where prefix size would otherwise exceed what the other providers handle cheaply. The cost to watch there is the hourly storage charge, not a write multiplier, and for a large enough document that storage cost disappears next to the savings on input tokens. Running through all of this is one variable that decides whether any of it pays off: the cache hit rate. A 95% hit rate on Anthropic captures close to the full 90% discount. A 60% hit rate captures far less, and at that lower rate, OpenAI's or Gemini's automatic caching may deliver a comparable or better result with none of the write-cost exposure Anthropic carries.

Common failure modes that silently destroy cache hit rates regardless of provider

Most caching failures don't announce themselves. The API still returns a normal response, the bill comes in higher than expected, and nothing in the output says the cache failed to fire. The most common mistake is placing dynamic content near the top of the prompt: a timestamp, a user ID, or any request-specific field sitting above the stable system prompt turns every single request into a full cache miss, because the match has to start at position zero. Static content goes first, dynamic content last, enforced at the application level and checked again every time the prompt template changes. Anthropic's four-breakpoint limit gives some flexibility here, letting a team isolate dynamic content between two static blocks, while OpenAI's automatic caching offers no equivalent control, leaving prompt ordering as the only lever available. Every provider's cache expires on a TTL, so a low-frequency workload keeps hitting cold starts between bursts of traffic and pays full price each time. That mismatch between how often requests arrive and how long the cache lasts is the root cause, and the fix is either increasing request frequency where that's possible, choosing a longer TTL tier if one is available, or simply building cold-start costs into the unit economics from the start. A third failure appears specifically in Anthropic's pricing: a prefix that only triggers one cache read before its TTL expires can end up costing more than it saves, given the write premium on the 1-hour tier. The 5-minute tier breaks even after just one read, the 1-hour tier after two, so teams running lower volumes are better off on the 5-minute tier, or on a provider with no write cost at all, like OpenAI's automatic caching or DeepSeek, until read frequency is established. A fourth and final failure is simply not checking whether any of this is working. All three major providers return cache hit data directly in the API response: Anthropic reports it as cache_read_input_tokens, OpenAI as cached_tokens inside prompt_tokens_details, and Gemini as cachedContentTokenCount. A team that never instruments or watches those fields has no way of knowing whether its caching strategy is doing anything at all, and a cache hit rate of zero on a workload that should be caching is a billing problem that stays invisible until someone goes looking for it.

Why a gateway should track, attribute, and protect prompt caching savings

Caching savings are fragile: without instrumentation, they are easy to lose without noticing. Without that layer, an organization sees raw metadata on each individual response and nothing resembling a real picture: what the true hit rate is, what a given feature costs, or which service is quietly bleeding savings through a badly structured prefix. A gateway sitting between applications and providers can surface cache-hit rates by service, model, and team in real time, turning cache performance into something a team can actually watch, rather than something reconstructed weeks later from an invoice. Running a multi-provider caching strategy, say, Anthropic for agentic work with long stable prefixes, OpenAI's automatic caching for simpler tasks, Gemini for large document corpora, stays manageable only through a single API that handles the provider-specific configuration underneath it. Concentrate AI, as an LLM gateway, is built to sit in that spot: a single API across providers, spend and cache performance visible by team and model, and caching configuration enforced centrally so new services inherit a working setup. Governance matters here too. The KV cache stores a representation of whatever prompt content it's given, so sensitive information sitting in a system prompt or a retrieved document stays on provider infrastructure for as long as the TTL lasts. Redacting personal or regulated data before it ever reaches the provider needs to happen at that central layer, not left to the judgment of whichever service happens to be making the call, so that what gets cached is already clean. Where zero-data-retention agreements are available, they shape what a provider caches and for how long, and any team with compliance requirements should confirm those terms before leaning on a long-TTL cache for workloads involving regulated data.