Guides / multi tenant llm cost
What tenancy does to LLM cost, and to prompt caching in particular
A prompt cache is keyed on content, so a system prompt every tenant shares still gets cached separately wherever the surrounding context differs. Cache cost then scales with tenant count rather than staying fixed. Culpa, a local-first LLM cost, margin, and forecast ledger, prices every call and attributes it to the tenant that made it.
Why this happens
Prompt caching is usually reasoned about as a fixed cost. You have a large system prompt, you cache it once, and every subsequent call reads it at a tenth of the input rate. That reasoning is correct for one tenant and quietly wrong for many. A cache is keyed on the content of what you cached, so two tenants sharing an identical system prompt still get separate cache entries whenever anything around it differs, which in a real multi-tenant application it always does: tenant configuration, retrieved documents, conversation history. What you have isn't one cache being read many times, it's many caches each being read a few times, and cache writes cost more than base input rather than less. The economics invert. At one tenant caching is the best saving available. At a few hundred tenants with light individual usage, you can be paying a premium on every call and calling it an optimisation.
What this usually looks like
- Cache write tokens are a large share of your input tokens.
- Your cache hit rate looks fine per tenant and poor overall.
- Cost per tenant is roughly flat regardless of how much each one uses.
- Caching was measured once, on one tenant, and never re-measured.
- Small tenants cost more per call than large ones.
Free, no card, no account
Run the free Cost Leak Scan
It shows your most expensive conversation before you install anything.
Mistakes that cost the most
| Mistake | Why it hurts | Do instead |
|---|---|---|
| Assuming a shared system prompt is cached once. | The cache is keyed on content, so differing surrounding context produces separate entries. | Measure cache writes against cache reads, and do it across tenants rather than within one. |
| Caching for tenants who don't call often enough. | A write costs more than plain input, so a cache read fewer times than it needs to break even loses money. | Cache for tenants above your break-even call rate and skip it below. |
| Reporting one blended cache hit rate. | It averages heavy tenants who benefit with light tenants who pay the premium. | Report hit rate per tenant, and look at the distribution. |
| Sharing one API key across tenants. | The invoice arrives as one number and no tenant can be priced from it. | Record the tenant on every call, which works at a finer grain than a key split does. |
Run this check tonight
- Work out your cache write tokens as a share of total input tokens.
- Calculate cache hit rate per tenant and look at the bottom quartile.
- Find how many calls a tenant needs before caching pays for itself, then count how many clear it.
- Check whether cost per call is higher for your smallest tenants.
The same system prompt, cached a hundred times
Illustrative example
A modelled 20,000-token system prompt shared by every tenant, cached on Claude Sonnet 5 at its real introductory rates from the price book effective 2026-07-02, where a 5-minute cache write is $2.50 per million against $2.00 base input. Each tenant's cache is written once an hour across a 30-day month. The prompt size, the tenant count and the refresh rate are modelled, the rates and the multipliers are published.
Nothing about the prompt changed and nothing about the pricing changed. The only variable is how many separate cache entries the same 20,000 tokens live in, and that number is your tenant count. This is the shape people miss when they reason about caching as a fixed cost, and it gets worse rather than better as the business grows.
Every number, with its confidence and source
| Figure | What it means | Confidence | Source |
|---|---|---|---|
| $3,600.00 against $36.00 per month | modelled cache-write cost for one shared system prompt across 100 tenants against one | calculated | A modelled 20,000-token system prompt on Claude Sonnet 5, where the 5-minute cache write rate is $2.50 per million from the price book effective 2026-07-02, being 1.25x the $2.00 base input rate Anthropic publishes. One write costs $0.0500, so hourly refreshes across 30 days cost $36.00 for one cache entry and $3,600.00 for 100. Rates and the cache multiplier are published, the prompt size, tenant count and refresh rate are modelled. |
What a generic answer can’t know
Which tenant made a call is your fact and nobody else's. A provider sees keys and models, and if every tenant shares a key it sees one customer making a lot of requests. Culpa records the tenant on each call alongside the customer, the feature and the run, prices it from a versioned price book in exact decimal, and keeps the ledger on infrastructure you control. That makes the questions tenancy actually raises answerable: cache hit rate per tenant rather than blended, which tenants sit below the break-even call rate for caching, and whether your smallest accounts cost more per call than your largest. Set it against what each tenant pays and the answer stops being a cost curve and becomes a pricing decision.
Questions founders ask next
Does prompt caching work across tenants?
Only where the cached content is genuinely identical. A cache is keyed on what you cached, so tenants sharing a system prompt still get separate entries whenever the surrounding context differs, which in a real multi-tenant app it usually does. You end up with many caches read a few times rather than one read many times.
Why does caching cost more as I add tenants?
Because writes scale with the number of cache entries and reads scale with usage. In the modelled example one 20,000-token cache refreshed hourly costs $36.00 a month, and the same prompt across 100 tenants costs $3,600.00, since it gets written 100 times rather than once.
When should I not cache?
Below the break-even call rate. A 5-minute cache write costs 1.25 times base input and a read costs a tenth, so caching pays after roughly one read within the window. A tenant calling less often than that pays a premium, and in a long tail of light tenants that adds up.
Is one API key per tenant enough to track this?
It splits the bill and stops there. A key tells you which tenant spent the money and nothing about which feature or conversation inside that tenant did. Recording the tenant on the call works at a finer grain and survives tenants sharing infrastructure.
On your infrastructure
Culpa runs on your infrastructure. Your prompts and responses never leave it. Culpa counts calls to run your plan, and it fails open, so if it ever breaks your app keeps running.
How Culpa works
Find the culprit. Not just the total.
Your dashboard shows what you spent. It stops short of who spent it. Culpa shows the conversation, the user and the feature behind it.
Your prompts stay local.
Culpa runs on your own infrastructure. What you send to a model reaches us at no point.
Every dollar has a name.
Follow any charge to the conversation, the user, the feature and the customer behind it.
See the bill before it lands.
Cost your next feature before you ship it. You get the likely bill and the worst case, at best, median, p90 and p99.
Three steps to your first answer.
Change one base URL.
Or drop in the Python or TypeScript library.
Find your most expensive conversation.
In the first session, not the first week.
Cost your next feature before you ship it.
Why the bill went up
Example dashboardCalls traced
418,209
across 3 projects
Spend this week
$378.41
+ $182 vs last week
Failed calls
312
74% retried, and you paid for all of them
+ $182 this week traced to one culprit
Spend over 14 days
Most expensive users
Next week forecast
Graded against reality. Accuracy shown as results land.
Keep reading
Sources: Anthropic, prompt caching, Anthropic pricing. Last reviewed 2026-08-05, rates effective 2026-07-02. Plain text version.