Guides / llm retry loop cost
What an LLM retry loop actually costs
A retry loop costs you the full rate on every attempt, because a provider bills tokens it generated even when your code discarded the response. Culpa, a local-first LLM cost, margin, and forecast ledger, meters failed and retried calls alongside successful ones, so your retry waste shows up as a number instead of an unexplained gap.
Why this happens
Retry logic is usually written for network failures, where a retry costs a round trip and nothing else. Model calls break that assumption. A call that times out after generating four hundred tokens bills for those four hundred tokens. A call that returns malformed JSON bills in full before your parser rejects it. So an exponential backoff with five attempts isn't a resilience setting, it's a spend multiplier of up to five.
What this usually looks like
- Spend spikes during provider incidents even though your successful request count falls.
- Your calculated cost from accepted responses is meaningfully below the invoice, and the gap widens under load.
- One feature with strict output parsing costs far more per successful result than its call count suggests.
- The bill rises on days your error dashboards were red, which nobody connects because the two live in different tools.
Free, no card, no account
Run the free Cost Leak Scan
It shows your most expensive conversation before you install anything.
Mistakes that cost the most
| Mistake | Why it hurts | Do instead |
|---|---|---|
| Reusing a network retry policy for model calls. | A failed HTTP request costs nothing. A failed model call can bill for a full response you never read. | Cap model retries at two, and only retry error classes that genuinely produced no output. |
| Retrying a malformed structured output at the same temperature. | The same prompt at the same settings tends to fail the same way, so you buy the identical failure repeatedly. | Change something on retry, the temperature or the instruction, or fall back to a cheaper repair pass. |
| Counting only successful calls in your cost model. | Your model then understates spend by exactly your retry rate, and it's most wrong during incidents. | Meter every attempt. Track attempts per successful result as a first-class number. |
Run this check tonight
- Count total attempts and divide by successful results. Anything meaningfully above one is retry spend.
- Check whether your retry path logs token usage. If it doesn't, that spend is invisible to your own cost model.
- Find your maximum retry count in code and multiply your worst-case call cost by it. That's your per-call ceiling.
- Look at spend on your last provider-incident day against a normal day, at equal successful volume.
- Confirm timeouts abort generation rather than letting the response finish unread. An abandoned response still bills.
A three-attempt retry on a structured-output call
A call on Claude Sonnet 5 at its real introductory rates, $0.002 per 1k input and $0.010 per 1k output. The prompt is 3,000 tokens and a valid response is 800 tokens. The parser rejects the first two attempts after the model has already produced a full response.
Three times the intended cost for one usable answer, and the two discarded attempts are invisible in any accounting that counts only successes.
Every number, with its confidence and source
| Figure | What it means | Confidence | Source |
|---|---|---|---|
| $0.042 | billed cost of one usable result after two rejected attempts | calculated | Three attempts at (3 x $0.002) + (0.8 x $0.010), using Claude Sonnet 5 introductory rates per 1k tokens from the price book, effective 2026-07-02. |
| 3x | multiple of intended cost paid for a single usable result at a three-attempt retry | calculated | $0.042 billed divided by the $0.014 single-attempt cost, from the teardown arithmetic. |
What a generic answer can’t know
A provider can't tell a retry from a fresh request. To it both are billable calls that succeeded. Only your own stack knows the second call exists because the first was rejected, and only your code knows the response was discarded. Culpa sees both on your infrastructure, where your prompts and responses stay, and counts the calls to run your plan.
Questions founders ask next
Do failed LLM calls still cost money?
If the model generated tokens before the failure, yes. A timeout partway through a response bills for what was produced. A response your parser rejects bills in full, because from the provider's side it succeeded.
What is a reasonable retry cap for model calls?
Two attempts covers genuine transient failures without turning a bad prompt into a spend multiplier. Beyond that you're usually paying repeatedly for a deterministic failure that a retry can't fix.
How do I find retry spend if my logs don't separate it?
Compare attempts to successful results at the call level. If your metering doesn't distinguish them, that's the first fix, because a retry rate you can't see is a cost you can't cap.
On your infrastructure
Culpa runs on your infrastructure. Your prompts and responses never leave it. Culpa counts calls to run your plan, and it fails open, so if it ever breaks your app keeps running.
Free, no card, no account
Run the free Cost Leak Scan
It shows your most expensive conversation before you install anything.
Keep reading
Sources: Anthropic pricing. Last reviewed 2026-08-01. Plain text version.