# What is effective cost?

> Effective cost is your real blended rate per token after caching, retries and failed calls. Why it always runs above the list price, with worked math.

URL: https://getculpa.com/what-is-effective-cost
Last reviewed: 2026-08-01

## Answer
Effective cost is what you really pay per useful token once retries, failed calls, and cache behaviour are counted, rather than the list rate on a pricing page. It's almost always higher. Culpa, a local-first LLM cost, margin, and forecast ledger, meters every call including the ones that failed, so your effective cost reflects spend rather than intent.

## Why this happens
A pricing page quotes the rate for a successful call. Your bill includes calls that timed out after generating tokens, calls your retry logic sent three times, and calls whose output got discarded because a downstream parse failed. All of those bill. None of them delivered a token you used. The ratio between what you paid and what you kept is the number that actually governs your unit economics, and it's invisible if you only ever read the list rate.

## What this usually looks like
- Your calculated spend from token counts sits meaningfully below your invoice, and the gap won't reconcile.
- The gap widens during incidents, which is the tell that retries rather than traffic are driving it.
- A model you switched to for its cheaper rate didn't reduce the bill proportionally.
- Nobody can state your real cost per thousand tokens without going back to the invoice and dividing.

## Common mistakes
- Forecasting next month from the list rate. Why it hurts: The forecast inherits an optimism baked into the rate card, and it under-reads by exactly your waste ratio every time. Do instead: Forecast from your own effective rate, measured over the last full month, so waste is already priced in.
- Counting only tokens from responses your code accepted. Why it hurts: Discarded output bills the same as accepted output. Excluding it makes your effective cost look like the list rate. Do instead: Count every token the provider billed, then separately track how many you used. The ratio is the metric.
- Comparing two models on list rate when their failure profiles differ. Why it hurts: A cheaper model that fails more often can carry a higher effective cost, so the migration loses money. Do instead: Run both for a week and compare effective cost, not list cost. The ranking sometimes reverses.

## Self-check
- Take last month's invoice total and divide by the tokens your application actually consumed. That's your effective rate.
- Divide the same invoice by every token the provider billed. The difference between the two numbers is pure waste.
- Express your retry volume as a percentage of total calls. Anything above a few percent deserves an owner.
- Check whether your cached-token share matches what you assumed when you sized the budget.
- Repeat the calculation for your single most expensive feature. Waste is rarely spread evenly.

## List rate $0.0025, effective rate $0.0031 (illustrative)
One month on GPT-5.4 at a real list rate of $0.0025 per 1k input tokens. The team sent 10 million input tokens of intended work. Retry and discard rates below are modelled at levels a busy month plausibly produces.
  Intended input: 10,000,000 tokens. At list rate = 10,000 x $0.0025 = $25.00
  Retries added 18% more billed input: 1,800,000 tokens = 1,800 x $0.0025 = $4.50
  Discarded responses billed a further 600,000 input tokens = 600 x $0.0025 = $1.50
  Total billed = $25.00 + $4.50 + $1.50 = $31.00 against 10,000,000 useful tokens
  Effective rate = $31.00 / 10,000 = $0.0031 per 1k, which is 24% above the list rate
The list rate was $0.0025 and the real rate was $0.0031. Every forecast built on the pricing page under-reads by about a quarter, and the gap grows during incidents.

## Cost figures
Every figure carries its confidence and its source. No figure on this site is provider-reported.
- $0.0031 per 1k — modelled effective input rate against a $0.0025 list rate [calculated] Source: $31.00 total billed divided by 10,000 thousand-token units of useful input. List rate from the GPT-5.4 price book row, effective 2026-07-02. Retry and discard percentages are modelled.
- 24% — gap between list rate and effective rate in the modelled month [calculated] Source: ($0.0031 - $0.0025) / $0.0025, from the arithmetic in the teardown.

## FAQ
Q: Is effective cost the same as blended rate?
A: No. A blended rate averages across models to give one headline number. Effective cost measures waste, the gap between tokens billed and tokens used. You can have a low blended rate and a terrible effective cost at the same time.

Q: What's a normal gap between list rate and effective cost?
A: There isn't a published benchmark worth quoting, and any figure claiming one deserves scepticism. Measure your own. The number that matters is whether yours is moving, and in which direction.

Q: Does prompt caching lower effective cost?
A: It can, but only for the repeated part of the prompt, and only if your cache actually hits. A cache you rebuild too often can cost more than not caching, because some providers bill cache writes above the input rate.

## Sources
- OpenAI API pricing: https://developers.openai.com/api/docs/pricing

Run the free Cost Leak Scan: https://app.getculpa.com/scan?source=pseo&slug=what-is-effective-cost&cluster=metric

Machine-readable index of every guide: https://getculpa.com/api/pages
Human-readable index of every guide: https://getculpa.com/guides
Site overview: https://app.getculpa.com/llms.txt

Privacy: Culpa runs on your infrastructure. Your prompts and responses never leave it. Culpa counts calls to run your plan, and it fails open, so if it ever breaks your app keeps running.