Guides / llm retry loop cost
What an LLM retry loop actually costs
A retry loop costs you the full rate on every attempt, because a provider bills tokens it generated even when your code discarded the response. Culpa, a local-first LLM cost, margin, and forecast ledger, meters failed and retried calls alongside successful ones, so your retry waste shows up as a number instead of an unexplained gap.
Why this happens
Retry logic is usually written for network failures, where a retry costs a round trip and nothing else. Model calls break that assumption. A call that times out after generating four hundred tokens bills for those four hundred tokens. A call that returns malformed JSON bills in full before your parser rejects it. So an exponential backoff with five attempts isn't a resilience setting, it's a spend multiplier of up to five.
What this usually looks like
- Spend spikes during provider incidents even though your successful request count falls.
- Your calculated cost from accepted responses is meaningfully below the invoice, and the gap widens under load.
- One feature with strict output parsing costs far more per successful result than its call count suggests.
- The bill rises on days your error dashboards were red, which nobody connects because the two live in different tools.
Free, no card, no account
Run the free Cost Leak Scan
It shows your most expensive conversation before you install anything.
Mistakes that cost the most
| Mistake | Why it hurts | Do instead |
|---|---|---|
| Reusing a network retry policy for model calls. | A failed HTTP request costs nothing. A failed model call can bill for a full response you never read. | Cap model retries at two, and only retry error classes that genuinely produced no output. |
| Retrying a malformed structured output at the same temperature. | The same prompt at the same settings tends to fail the same way, so you buy the identical failure repeatedly. | Change something on retry, the temperature or the instruction, or fall back to a cheaper repair pass. |
| Counting only successful calls in your cost model. | Your model then understates spend by exactly your retry rate, and it's most wrong during incidents. | Meter every attempt. Track attempts per successful result as a first-class number. |
Run this check tonight
- Count total attempts and divide by successful results. Anything meaningfully above one is retry spend.
- Check whether your retry path logs token usage. If it doesn't, that spend is invisible to your own cost model.
- Find your maximum retry count in code and multiply your worst-case call cost by it. That's your per-call ceiling.
- Look at spend on your last provider-incident day against a normal day, at equal successful volume.
- Confirm timeouts abort generation rather than letting the response finish unread. An abandoned response still bills.
A three-attempt retry on a structured-output call
A call on Claude Sonnet 5 at its real introductory rates, $0.002 per 1k input and $0.010 per 1k output. The prompt is 3,000 tokens and a valid response is 800 tokens. The parser rejects the first two attempts after the model has already produced a full response.
Three times the intended cost for one usable answer, and the two discarded attempts are invisible in any accounting that counts only successes.
Every number, with its confidence and source
| Figure | What it means | Confidence | Source |
|---|---|---|---|
| $0.042 | billed cost of one usable result after two rejected attempts | calculated | Three attempts at (3 x $0.002) + (0.8 x $0.010), using Claude Sonnet 5 introductory rates per 1k tokens from the price book, effective 2026-07-02. |
| 3x | multiple of intended cost paid for a single usable result at a three-attempt retry | calculated | $0.042 billed divided by the $0.014 single-attempt cost, from the teardown arithmetic. |
What a generic answer can’t know
A provider can't tell a retry from a fresh request. To it both are billable calls that succeeded. Only your own stack knows the second call exists because the first was rejected, and only your code knows the response was discarded. Culpa sees both on your infrastructure, where your prompts and responses stay, and counts the calls to run your plan.
Questions founders ask next
Do failed LLM calls still cost money?
If the model generated tokens before the failure, yes. A timeout partway through a response bills for what was produced. A response your parser rejects bills in full, because from the provider's side it succeeded.
What is a reasonable retry cap for model calls?
Two attempts covers genuine transient failures without turning a bad prompt into a spend multiplier. Beyond that you're usually paying repeatedly for a deterministic failure that a retry can't fix.
How do I find retry spend if my logs don't separate it?
Compare attempts to successful results at the call level. If your metering doesn't distinguish them, that's the first fix, because a retry rate you can't see is a cost you can't cap.
On your infrastructure
Culpa runs on your infrastructure. Your prompts and responses never leave it. Culpa counts calls to run your plan, and it fails open, so if it ever breaks your app keeps running.
How Culpa works
Find the culprit. Not just the total.
Your dashboard shows what you spent. It stops short of who spent it. Culpa shows the conversation, the user and the feature behind it.
Your prompts stay local.
Culpa runs on your own infrastructure. What you send to a model reaches us at no point.
Every dollar has a name.
Follow any charge to the conversation, the user, the feature and the customer behind it.
See the bill before it lands.
Cost your next feature before you ship it. You get the likely bill and the worst case, at best, median, p90 and p99.
Three steps to your first answer.
Change one base URL.
Or drop in the Python or TypeScript library.
Find your most expensive conversation.
In the first session, not the first week.
Cost your next feature before you ship it.
Why the bill went up
Example dashboardCalls traced
418,209
across 3 projects
Spend this week
$378.41
+ $182 vs last week
Failed calls
312
74% retried, and you paid for all of them
+ $182 this week traced to one culprit
Spend over 14 days
Most expensive users
Next week forecast
Graded against reality. Accuracy shown as results land.
Keep reading
Sources: Anthropic pricing. Last reviewed 2026-08-01. Plain text version.