Guides / prompt cache miss cost
What a prompt cache miss costs you
A prompt cache miss bills your repeated prompt at the full input rate instead of the cached rate, which on some models is ten times more. Culpa, a local-first LLM cost, margin, and forecast ledger, meters cached, uncached and cache-write tokens separately per call, so a cache that stopped working shows up the day it breaks.
Why this happens
Caching matches on prefixes, so it works only when the repeated block sits at the very start of the prompt and stays byte-identical. Three things break that. A timestamp or a user name injected before the system prompt. A deploy that edits the cached block and invalidates every prefix at once. And an expiry window shorter than your traffic gap, so the cache is rebuilt more often than it's read.
What this usually looks like
- Your cached-token share sits near zero even though the same system prompt goes out on every call.
- Cache hits collapse immediately after every deploy and recover slowly, or never.
- Spend rose slightly after you switched caching on, which is the signature of writes outnumbering reads.
- Hit rate is fine during busy hours and terrible overnight, because the cache expires between calls.
Free, no card, no account
Run the free Cost Leak Scan
It shows your most expensive conversation before you install anything.
Mistakes that cost the most
| Mistake | Why it hurts | Do instead |
|---|---|---|
| Putting anything variable before the stable block. | A prefix match fails at the first differing token, so one leading timestamp makes the entire prompt uncacheable. | Order the prompt stable first and variable last. That single reordering is usually the whole fix. |
| Treating the hit rate as the only number that matters. | On Anthropic models a cache write bills above the input rate, so a high write rate loses money at any hit rate. | Track writes against reads. A five-minute cache pays off on the first read, so a write never read is the only losing case. |
| Editing the cached system prompt as part of routine tuning. | Every edit invalidates the prefix for all traffic at once, so the saving resets on each deploy. | Freeze the cached block and put anything you tune often after it, where changes are free. |
Run this check tonight
- Divide cached input tokens by total input tokens. That percentage is your hit rate.
- Compare the first 200 characters of your prompt across ten consecutive calls. Any difference explains a zero hit rate.
- Count cache writes against cache reads for the same window. More writes than reads means the cache is a cost.
- Plot hit rate against your deploy timestamps. Cliffs at deploys mean the cached block is being edited.
- Check your quietest hour. If hits vanish there, your traffic gap exceeds the cache window.
A 4,000-token system prompt, cached and missed
Illustrative example
Claude Haiku 4.5 at real rates from the price book: $0.001 per 1k input, $0.0001 per 1k cached read, and a five-minute cache write derived at 1.25 times input, so $0.00125 per 1k. One million calls a month.
The gap between a working cache and a broken one is $3,600 a month at this volume. A cache rebuilt every call is worse than no cache at all, by a quarter.
Every number, with its confidence and source
| Figure | What it means | Confidence | Source |
|---|---|---|---|
| 10x | cost of a cache miss against a cache hit on Claude Haiku 4.5 | calculated | $0.001 input rate divided by the $0.0001 cached rate, both per 1k tokens from the Claude Haiku 4.5 price book row, effective 2026-07-02. |
| $400 to $4,000 | modelled monthly cost of a 4,000-token system prompt at one million calls, fully cached versus fully missed | estimated | Endpoints from the teardown arithmetic at real Claude Haiku 4.5 rates. A range because it brackets the two extremes of hit rate. |
What a generic answer can’t know
A provider reports cached tokens on a response, but it can't tell you why the hit rate is low, because the reason lives in how your application assembles prompts. Diagnosing it means comparing prompt structure across calls. Culpa does that on your infrastructure, where your prompts and responses stay, and counts the calls to run your plan.
Questions founders ask next
Why is my prompt cache hit rate zero?
Almost always something variable sits before the stable block. A timestamp, a user identifier or a randomised instruction at the front breaks the prefix match, and everything after it becomes uncacheable no matter how repetitive it's.
Can prompt caching cost more than it saves?
Yes, on providers that bill cache writes. In this price book only Anthropic models carry a cache-write rate, at 1.25 times input for the five-minute window and 2 times for the one-hour window. Write more often than you read and you lose money.
Does caching help with output tokens?
No. Caching applies to input only. If your bill is dominated by long completions, caching is the wrong lever entirely and shortening the output is where the money is.
On your infrastructure
Culpa runs on your infrastructure. Your prompts and responses never leave it. Culpa counts calls to run your plan, and it fails open, so if it ever breaks your app keeps running.
Free, no card, no account
Run the free Cost Leak Scan
It shows your most expensive conversation before you install anything.
Keep reading
Sources: Anthropic pricing. Last reviewed 2026-08-01. Plain text version.