Guides / effective cost
What is effective cost?
Effective cost is what you really pay per useful token once retries, failed calls, and cache behaviour are counted, rather than the list rate on a pricing page. It's almost always higher. Culpa, a local-first LLM cost, margin, and forecast ledger, meters every call including the ones that failed, so your effective cost reflects spend rather than intent.
Why this happens
A pricing page quotes the rate for a successful call. Your bill includes calls that timed out after generating tokens, calls your retry logic sent three times, and calls whose output got discarded because a downstream parse failed. All of those bill. None of them delivered a token you used. The ratio between what you paid and what you kept is the number that actually governs your unit economics, and it's invisible if you only ever read the list rate.
What this usually looks like
- Your calculated spend from token counts sits meaningfully below your invoice, and the gap won't reconcile.
- The gap widens during incidents, which is the tell that retries rather than traffic are driving it.
- A model you switched to for its cheaper rate didn't reduce the bill proportionally.
- Nobody can state your real cost per thousand tokens without going back to the invoice and dividing.
Free, no card, no account
Run the free Cost Leak Scan
It shows your most expensive conversation before you install anything.
Mistakes that cost the most
| Mistake | Why it hurts | Do instead |
|---|---|---|
| Forecasting next month from the list rate. | The forecast inherits an optimism baked into the rate card, and it under-reads by exactly your waste ratio every time. | Forecast from your own effective rate, measured over the last full month, so waste is already priced in. |
| Counting only tokens from responses your code accepted. | Discarded output bills the same as accepted output. Excluding it makes your effective cost look like the list rate. | Count every token the provider billed, then separately track how many you used. The ratio is the metric. |
| Comparing two models on list rate when their failure profiles differ. | A cheaper model that fails more often can carry a higher effective cost, so the migration loses money. | Run both for a week and compare effective cost, not list cost. The ranking sometimes reverses. |
Run this check tonight
- Take last month's invoice total and divide by the tokens your application actually consumed. That's your effective rate.
- Divide the same invoice by every token the provider billed. The difference between the two numbers is pure waste.
- Express your retry volume as a percentage of total calls. Anything above a few percent deserves an owner.
- Check whether your cached-token share matches what you assumed when you sized the budget.
- Repeat the calculation for your single most expensive feature. Waste is rarely spread evenly.
List rate $0.0025, effective rate $0.0031
Illustrative example
One month on GPT-5.4 at a real list rate of $0.0025 per 1k input tokens. The team sent 10 million input tokens of intended work. Retry and discard rates below are modelled at levels a busy month plausibly produces.
The list rate was $0.0025 and the real rate was $0.0031. Every forecast built on the pricing page under-reads by about a quarter, and the gap grows during incidents.
Every number, with its confidence and source
| Figure | What it means | Confidence | Source |
|---|---|---|---|
| $0.0031 per 1k | modelled effective input rate against a $0.0025 list rate | calculated | $31.00 total billed divided by 10,000 thousand-token units of useful input. List rate from the GPT-5.4 price book row, effective 2026-07-02. Retry and discard percentages are modelled. |
| 24% | gap between list rate and effective rate in the modelled month | calculated | ($0.0031 - $0.0025) / $0.0025, from the arithmetic in the teardown. |
What a generic answer can’t know
A provider knows what it billed you. It doesn't know which of those tokens you threw away, because usefulness is a judgement your application makes after the response arrives. Only something watching both sides, the request and what your code did with it, can compute an effective rate. Culpa does that on your infrastructure, where your prompts and responses stay, and counts the calls to run your plan.
Questions founders ask next
Is effective cost the same as blended rate?
No. A blended rate averages across models to give one headline number. Effective cost measures waste, the gap between tokens billed and tokens used. You can have a low blended rate and a terrible effective cost at the same time.
What's a normal gap between list rate and effective cost?
There isn't a published benchmark worth quoting, and any figure claiming one deserves scepticism. Measure your own. The number that matters is whether yours is moving, and in which direction.
Does prompt caching lower effective cost?
It can, but only for the repeated part of the prompt, and only if your cache actually hits. A cache you rebuild too often can cost more than not caching, because some providers bill cache writes above the input rate.
On your infrastructure
Culpa runs on your infrastructure. Your prompts and responses never leave it. Culpa counts calls to run your plan, and it fails open, so if it ever breaks your app keeps running.
Free, no card, no account
Run the free Cost Leak Scan
It shows your most expensive conversation before you install anything.
Keep reading
Sources: OpenAI API pricing. Last reviewed 2026-08-01. Plain text version.