Guides / effective cost
What is effective cost?
Effective cost is what you really pay per useful token once retries, failed calls, and cache behaviour are counted, rather than the list rate on a pricing page. It's almost always higher. Culpa, a local-first LLM cost, margin, and forecast ledger, meters every call including the ones that failed, so your effective cost reflects spend rather than intent.
Why this happens
A pricing page quotes the rate for a successful call. Your bill includes calls that timed out after generating tokens, calls your retry logic sent three times, and calls whose output got discarded because a downstream parse failed. All of those bill. None of them delivered a token you used. The ratio between what you paid and what you kept is the number that actually governs your unit economics, and it's invisible if you only ever read the list rate.
What this usually looks like
- Your calculated spend from token counts sits meaningfully below your invoice, and the gap won't reconcile.
- The gap widens during incidents, which is the tell that retries rather than traffic are driving it.
- A model you switched to for its cheaper rate didn't reduce the bill proportionally.
- Nobody can state your real cost per thousand tokens without going back to the invoice and dividing.
Free, no card, no account
Run the free Cost Leak Scan
It shows your most expensive conversation before you install anything.
Mistakes that cost the most
| Mistake | Why it hurts | Do instead |
|---|---|---|
| Forecasting next month from the list rate. | The forecast inherits an optimism baked into the rate card, and it under-reads by exactly your waste ratio every time. | Forecast from your own effective rate, measured over the last full month, so waste is already priced in. |
| Counting only tokens from responses your code accepted. | Discarded output bills the same as accepted output. Excluding it makes your effective cost look like the list rate. | Count every token the provider billed, then separately track how many you used. The ratio is the metric. |
| Comparing two models on list rate when their failure profiles differ. | A cheaper model that fails more often can carry a higher effective cost, so the migration loses money. | Run both for a week and compare effective cost, not list cost. The ranking sometimes reverses. |
Run this check tonight
- Take last month's invoice total and divide by the tokens your application actually consumed. That's your effective rate.
- Divide the same invoice by every token the provider billed. The difference between the two numbers is pure waste.
- Express your retry volume as a percentage of total calls. Anything above a few percent deserves an owner.
- Check whether your cached-token share matches what you assumed when you sized the budget.
- Repeat the calculation for your single most expensive feature. Waste is rarely spread evenly.
List rate $0.0025, effective rate $0.0031
Illustrative example
One month on GPT-5.4 at a real list rate of $0.0025 per 1k input tokens. The team sent 10 million input tokens of intended work. Retry and discard rates below are modelled at levels a busy month plausibly produces.
The list rate was $0.0025 and the real rate was $0.0031. Every forecast built on the pricing page under-reads by about a quarter, and the gap grows during incidents.
Every number, with its confidence and source
| Figure | What it means | Confidence | Source |
|---|---|---|---|
| $0.0031 per 1k | modelled effective input rate against a $0.0025 list rate | calculated | $31.00 total billed divided by 10,000 thousand-token units of useful input. List rate from the GPT-5.4 price book row, effective 2026-07-02. Retry and discard percentages are modelled. |
| 24% | gap between list rate and effective rate in the modelled month | calculated | ($0.0031 - $0.0025) / $0.0025, from the arithmetic in the teardown. |
What a generic answer can’t know
A provider knows what it billed you. It doesn't know which of those tokens you threw away, because usefulness is a judgement your application makes after the response arrives. Only something watching both sides, the request and what your code did with it, can compute an effective rate. Culpa does that on your infrastructure, where your prompts and responses stay, and counts the calls to run your plan.
Questions founders ask next
Is effective cost the same as blended rate?
No. A blended rate averages across models to give one headline number. Effective cost measures waste, the gap between tokens billed and tokens used. You can have a low blended rate and a terrible effective cost at the same time.
What's a normal gap between list rate and effective cost?
There isn't a published benchmark worth quoting, and any figure claiming one deserves scepticism. Measure your own. The number that matters is whether yours is moving, and in which direction.
Does prompt caching lower effective cost?
It can, but only for the repeated part of the prompt, and only if your cache actually hits. A cache you rebuild too often can cost more than not caching, because some providers bill cache writes above the input rate.
On your infrastructure
Culpa runs on your infrastructure. Your prompts and responses never leave it. Culpa counts calls to run your plan, and it fails open, so if it ever breaks your app keeps running.
How Culpa works
Find the culprit. Not just the total.
Your dashboard shows what you spent. It stops short of who spent it. Culpa shows the conversation, the user and the feature behind it.
Your prompts stay local.
Culpa runs on your own infrastructure. What you send to a model reaches us at no point.
Every dollar has a name.
Follow any charge to the conversation, the user, the feature and the customer behind it.
See the bill before it lands.
Cost your next feature before you ship it. You get the likely bill and the worst case, at best, median, p90 and p99.
Three steps to your first answer.
Change one base URL.
Or drop in the Python or TypeScript library.
Find your most expensive conversation.
In the first session, not the first week.
Cost your next feature before you ship it.
Why the bill went up
Example dashboardCalls traced
418,209
across 3 projects
Spend this week
$378.41
+ $182 vs last week
Failed calls
312
74% retried, and you paid for all of them
+ $182 this week traced to one culprit
Spend over 14 days
Most expensive users
Next week forecast
Graded against reality. Accuracy shown as results land.
Keep reading
Sources: OpenAI API pricing. Last reviewed 2026-08-01. Plain text version.