Guides / llm cost per deploy
Attributing an LLM cost change to the deploy that caused it
Tracking LLM cost per deploy means recording which release produced each call, then comparing cost per call across releases rather than comparing monthly totals. Traffic and code change at the same time, so a total can't separate them. Culpa, a local-first LLM cost, margin, and forecast ledger, tags every call with its release and prices it individually.
Why this happens
A deploy is the most common cause of a cost change and the hardest to prove, because two things move at once. The release alters cost per call, through a longer prompt, an extra retrieval step, a different model or one more retry. Meanwhile traffic does whatever traffic does. The monthly total mixes them, and mixing isn't reversible: a 15% rise in unit cost alongside a 10% fall in volume shows up as a 3.5% rise in the total, which reads as noise and gets ignored. Worse, the two can cancel completely, and then a genuine regression leaves no trace at all. The fix is unglamorous and it works. Record the release identifier on every call, then compare cost per call between releases, which holds volume constant by construction. That comparison is only available if the tag was written at the time. Adding release tracking after a suspicious month tells you about next month, never about the one you're investigating.
What this usually looks like
- Cost rose after a release and nobody can prove the release caused it.
- Your bill looks flat and cost per call has been climbing for three deploys.
- A rollback fixed spend and nothing recorded which version was responsible.
- You compare months, and deploys happen weekly.
- Two changes shipped together and only one of them was the expensive one.
Free, no card, no account
Run the free Cost Leak Scan
It shows your most expensive conversation before you install anything.
Mistakes that cost the most
| Mistake | Why it hurts | Do instead |
|---|---|---|
| Comparing monthly totals across a release. | Traffic and unit cost both move, and a total can't tell you which one did. | Compare cost per call between releases, which holds volume constant by construction. |
| Adding release tracking after a suspicious month. | The tag is written at call time, so it can never describe calls that already happened. | Tag from now on, and accept that the current investigation stays a judgement call. |
| Shipping several changes in one release and pricing the release. | You learn the release cost more without learning which change did it. | Where cost matters, ship the cost-affecting change alone and measure it. |
| Treating a cost regression as a bug rather than a decision. | A longer prompt that improves quality is a trade, and it should be priced rather than reverted on reflex. | Put the unit-cost change beside the quality change and decide deliberately. |
Run this check tonight
- Find your last three releases and your cost per call in the week either side of each.
- If you can't produce that, your ledger has no release tag and that's the finding.
- Take your most recent unexplained cost movement and see whether a deploy sits at its start.
- Check whether your release identifier reaches calls made inside libraries you didn't write.
A 15% regression that looks like 3.5%
Illustrative example
A modelled service at 2,000,000 calls a month, priced on Claude Haiku 4.5 at real rates of $1.00 and $5.00 per million from the price book effective 2026-07-02. A release adds retrieval context, taking input from 2,000 to 2,600 tokens a call while output holds at 400. In the same month, traffic falls 10% for unrelated reasons. Token shapes and volumes are modelled.
A 3.5% monthly movement is inside the range nobody investigates. The regression behind it runs more than four times larger, and unlike the traffic fall it sticks. It stays invisible for as long as the comparison is made on totals.
Every number, with its confidence and source
| Figure | What it means | Confidence | Source |
|---|---|---|---|
| $280.00 | modelled monthly movement from a 15% unit-cost regression, masked by a 10% traffic fall | calculated | Per call, 2,000 input and 400 output tokens cost $0.0040, and 2,600 input and 400 output cost $0.0046, both on Claude Haiku 4.5 at real rates of $1.00 and $5.00 per million from the price book effective 2026-07-02, a 15.0% rise. At 2,000,000 calls before and 1,800,000 after, $8,000.00 against $8,280.00, a $280.00 or 3.5% movement. Token shapes and call volumes are modelled. |
What a generic answer can’t know
A release identifier is application knowledge. It exists in your deploy pipeline and nowhere near a provider invoice, which carries no idea that your releases exist. Nothing downstream can reconstruct it, so the tag has to be attached at call time, because one written later describes nothing earlier. Culpa records the release alongside the model version, the prompt version, the customer and the run, prices each call from a versioned price book in exact decimal, and keeps the ledger on your own infrastructure so a comparison across releases stays available long after the deploy. That turns cost per call by release into a query rather than an investigation, and it makes a regression a decision you priced rather than one you inherited.
Questions founders ask next
How do I tell whether a deploy raised my LLM costs?
Compare cost per call between releases rather than totals between months. A total mixes unit cost with traffic and the mix isn't reversible. Cost per call holds volume constant, which is what makes the difference attributable to the code.
Why can a 15% regression look like a 3.5% rise?
Because traffic moves too. In the modelled example a 15% rise in cost per call landed alongside a 10% fall in volume, so the monthly total rose only 3.5%, which reads as noise. The two can also cancel exactly, leaving a real regression with no trace in the total at all.
Can I add release tracking retroactively?
Not exactly. A time-based join to your deploy timestamps approximates it, and doing so is worthwhile. The tag itself is written when the call is made, so it only describes calls from that point forward. Worth setting up before you need it, because the moment you want it always lands during an investigation into something that already happened.
What should I record on each call?
The release or commit identifier, the model version, the prompt version, the customer and the run. Those five make almost every cost question a query rather than a reconstruction, and they cost almost nothing to carry compared to the trace payload most platforms bill you to store.
On your infrastructure
Culpa runs on your infrastructure. Your prompts and responses never leave it. Culpa counts calls to run your plan, and it fails open, so if it ever breaks your app keeps running.
How Culpa works
Find the culprit. Not just the total.
Your dashboard shows what you spent. It stops short of who spent it. Culpa shows the conversation, the user and the feature behind it.
Your prompts stay local.
Culpa runs on your own infrastructure. What you send to a model reaches us at no point.
Every dollar has a name.
Follow any charge to the conversation, the user, the feature and the customer behind it.
See the bill before it lands.
Cost your next feature before you ship it. You get the likely bill and the worst case, at best, median, p90 and p99.
Three steps to your first answer.
Change one base URL.
Or drop in the Python or TypeScript library.
Find your most expensive conversation.
In the first session, not the first week.
Cost your next feature before you ship it.
Why the bill went up
Example dashboardCalls traced
418,209
across 3 projects
Spend this week
$378.41
+ $182 vs last week
Failed calls
312
74% retried, and you paid for all of them
+ $182 this week traced to one culprit
Spend over 14 days
Most expensive users
Next week forecast
Graded against reality. Accuracy shown as results land.
Keep reading
Sources: Anthropic pricing. Last reviewed 2026-08-03, rates effective 2026-07-02. Plain text version.