Guides / context overflow cost
What to do when a conversation outgrows its context window
A conversation that fills its window has to lose something, and the two options cost differently. Truncating is free and loses information. Summarising costs one call and cuts every later one. Culpa, a local-first LLM cost, margin, and forecast ledger, prices each call so the break-even is measured rather than guessed.
Why this happens
Long-running conversations grow until they hit a limit, and something has to give. Truncating drops the oldest turns, costs nothing to do, and loses whatever was in them, which tends to be the setup the later turns depend on. Summarising compresses the history into a shorter form, keeps most of the meaning, and costs a model call to produce. Teams usually pick by instinct, and instinct says the free option is cheaper. It generally isn't, because the summary is paid once and the saving repeats on every subsequent call in the conversation. The break-even is low enough to be worth computing rather than debating, and it gets lower the more expensive the model serving the conversation is, because the summary can be produced on a cheap model while the saving accrues at the expensive one's input rate. That asymmetry is the whole trick and it's rarely used deliberately.
What this usually looks like
- Conversations hit the window limit and the handling was never designed.
- You truncate because it's free and nobody priced the alternative.
- Cost per call climbs through a long session with no ceiling.
- Summaries are generated on the same expensive model serving the conversation.
- Users report the assistant forgetting things it was told earlier.
Free, no card, no account
Run the free Cost Leak Scan
It shows your most expensive conversation before you install anything.
Mistakes that cost the most
| Mistake | Why it hurts | Do instead |
|---|---|---|
| Truncating because it costs nothing to do. | It costs nothing once and loses context permanently, while a summary pays for itself in a call or two. | Compute the break-even for your own conversation lengths and pick deliberately. |
| Summarising on the model serving the conversation. | The summary is a mechanical task that a cheap model does well, so an expensive one wastes the saving. | Produce the summary on the cheapest capable model and keep the saving. |
| Waiting for the window to fill before acting. | Every call before the limit already carried the full history at full input rates. | Compress on a threshold you choose, not on the one the provider enforces. |
| Summarising once and letting it grow again. | The problem recurs, and a second overflow costs the same as the first. | Treat compression as a recurring step with its own budget, not a one-off repair. |
Run this check tonight
- Find how many of your conversations reach the window limit.
- Work out the average number of calls that happen after a conversation passes your threshold.
- Compare that to the break-even for summarising at your own token counts.
- Check which model your summaries are generated on.
Summarise once, save on every call after
Illustrative example
A modelled 100,000-token conversation compressed to 20,000, with the summary produced on Claude Haiku 4.5 and the conversation itself served by Claude Sonnet 5. Real rates from the price book effective 2026-07-02: Haiku at $1.00 and $5.00 per million, Sonnet 5 at its introductory $2.00 input. Token counts and the compression ratio are modelled, the rates are published.
One and a quarter calls. Any conversation that continues past a second turn after compressing is already ahead, and the saving repeats for as long as the session runs. Producing the summary on the cheap model is what makes the break-even that low: the same summary generated on Sonnet 5 would cost more and recover it more slowly.
Every number, with its confidence and source
| Figure | What it means | Confidence | Source |
|---|---|---|---|
| 1.25 calls | modelled break-even for summarising a 100,000-token conversation rather than truncating it | calculated | A modelled 100,000-token conversation compressed to 20,000 on Claude Haiku 4.5 at real rates of $1.00 and $5.00 per million from the price book effective 2026-07-02 costs $0.1000 + $0.1000 = $0.2000. The 80,000 tokens saved on each later call, served by Claude Sonnet 5 at its real introductory $2.00 per million, is worth $0.1600 a call. $0.2000 / $0.1600 = 1.25 calls. Rates are published, the token counts and compression ratio are modelled. |
What a generic answer can’t know
How long your conversations run, how many continue past a compression point, and how much of the history is genuinely reusable are all facts about your own sessions. A pricing page can give you the rates and nothing else in this calculation. Culpa prices every call from a versioned price book in exact decimal and groups calls by conversation, so the break-even is measured against your real distribution rather than an assumed one, and the sessions that never reach it are visible as their own group. That matters because compressing a conversation that ends immediately afterwards is pure cost, and the only way to know how often that happens is to have counted it.
Questions founders ask next
Is it cheaper to truncate or to summarise?
Summarising, in most cases, and the break-even is low. In the modelled example a summary costs $0.2000 once and saves $0.1600 on every later call, so it pays for itself after 1.25 calls. Truncating costs nothing and loses the context permanently.
Which model should generate the summary?
The cheapest one that does it well. Summarising is mechanical, and the saving accrues at the expensive model's input rate regardless of what produced the summary. In the modelled example generating on Haiku 4.5 while serving on Sonnet 5 is what puts the break-even at 1.25 calls.
When should I compress?
At a threshold you choose rather than the one the window enforces. Every call before you act already carried the full history at full input rates, so waiting for the limit means paying for the problem the whole way up to it.
Does compressing ever lose money?
Yes, on conversations that end immediately after. The summary is paid once and recovers over subsequent calls, so a session that stops before the break-even is pure cost. Whether that happens often enough to matter is measurable on your own sessions and not guessable from anything else.
On your infrastructure
Culpa runs on your infrastructure. Your prompts and responses never leave it. Culpa counts calls to run your plan, and it fails open, so if it ever breaks your app keeps running.
How Culpa works
Find the culprit. Not just the total.
Your dashboard shows what you spent. It stops short of who spent it. Culpa shows the conversation, the user and the feature behind it.
Your prompts stay local.
Culpa runs on your own infrastructure. What you send to a model reaches us at no point.
Every dollar has a name.
Follow any charge to the conversation, the user, the feature and the customer behind it.
See the bill before it lands.
Cost your next feature before you ship it. You get the likely bill and the worst case, at best, median, p90 and p99.
Three steps to your first answer.
Change one base URL.
Or drop in the Python or TypeScript library.
Find your most expensive conversation.
In the first session, not the first week.
Cost your next feature before you ship it.
Why the bill went up
Example dashboardCalls traced
418,209
across 3 projects
Spend this week
$378.41
+ $182 vs last week
Failed calls
312
74% retried, and you paid for all of them
+ $182 this week traced to one culprit
Spend over 14 days
Most expensive users
Next week forecast
Graded against reality. Accuracy shown as results land.
Keep reading
Sources: Anthropic pricing. Last reviewed 2026-08-05, rates effective 2026-07-02. Plain text version.