Guides / track llm cost by prompt version
How to track LLM cost by prompt version
To track LLM cost by prompt version you record which version produced each call, then price the calls and compare versions on cost per call rather than on total spend. Culpa, a local-first LLM cost, margin, and forecast ledger, tags every call with its version and prices it from a versioned price book, so a prompt change and a rate change stay separable.
Why this happens
A prompt edit is a deploy that changes your unit cost, and almost nobody treats it as one. Add two sentences of instruction to a system prompt and every call from that moment carries them, so a change that took a minute to write raises cost for as long as it ships. The reason it goes unnoticed is that the total moves for several reasons at once. Traffic grew, a provider changed a rate, someone switched a model, and someone edited a prompt, and a single monthly number can't separate those. Tagging each call with the version it came from separates them completely, because cost per call under version 4 against version 3 holds traffic constant by construction. Without that tag, the honest answer to why the bill moved is that nobody knows, and the usual response is to blame volume.
What this usually looks like
- Cost per call rose and traffic didn't, and nobody can name the change.
- A prompt was edited last month and no record connects it to the bill.
- You compare prompt versions on quality and never on cost.
- A rollback fixed spend and nobody can prove which version was responsible.
- Your prompts are versioned in one system and your spend lives in another.
Free, no card, no account
Run the free Cost Leak Scan
It shows your most expensive conversation before you install anything.
Mistakes that cost the most
| Mistake | Why it hurts | Do instead |
|---|---|---|
| Comparing prompt versions on total spend. | Totals move with traffic. Two versions running at different volumes can't be compared that way. | Compare cost per call, which holds volume constant by construction. |
| Treating a prompt edit as content rather than a deploy. | It changes unit economics on every call afterwards and usually ships without review. | Version prompts, tag calls with the version, and price the change before it goes out. |
| Tagging only the calls your own code makes. | Framework retries and sub-calls run the same prompt and land untagged, so the version's real cost is understated. | Confirm on a live trace that every call carrying that prompt also carries the version. |
| Assuming a longer prompt costs proportionally more. | It depends where the tokens land. Input is usually several times cheaper than output. | Price the actual input and output split rather than reasoning from prompt length. |
Run this check tonight
- Name the current version of your highest-volume prompt, and when it last changed.
- Find the cost per call of the version before it.
- Add up the tokens your system prompt contributes to a single call.
- Multiply that by last month's call count and decide whether the wording was worth it.
What two sentences cost at volume
A modelled system prompt gains 180 tokens of extra instruction, which is roughly two careful sentences with an example. Priced on Claude Haiku 4.5 at a real input rate of $1.00 per million from the price book, effective 2026-07-02, across a modelled 4 million calls a month. Nothing else about the workload changes.
The per-call number is too small to notice and the monthly number is a hire. That gap is the entire argument for versioning: the change is undetectable at the granularity people look at, and material at the granularity finance uses.
Every number, with its confidence and source
| Figure | What it means | Confidence | Source |
|---|---|---|---|
| $720.00 per month | modelled cost of adding 180 input tokens to a system prompt at 4M calls a month | calculated | 180 tokens x 4,000,000 calls = 720,000,000 input tokens, at the real Claude Haiku 4.5 input rate of $1.00 per million from the price book effective 2026-07-02, giving $720.00. The token increment and the call volume are modelled. The rate is real. |
| $0.00018 | the same change expressed per call, where it disappears into the noise | calculated | 180 tokens at $1.00 per million = $0.00018 per call, from the same real rate. Shown beside the monthly figure because the pair is the finding: the same change is invisible at one granularity and material at another. |
What a generic answer can’t know
A prompt management tool knows which version is deployed. A provider invoice knows what the month cost. Nothing joins them unless the version travels on the call, and by the time the invoice arrives the context is gone for good. Culpa records the version at capture, prices the call from a price book carried with effective dates, and stores money as exact decimal, which is what lets it separate the two things that move a bill: your change and the provider's. Ask it what version 4 costs per call against version 3 and the comparison holds traffic constant. Ask it what the same version cost before and after a rate change and it recomputes each month at the rate in force when the calls ran.
Questions founders ask next
How do I compare the cost of two prompt versions?
On cost per call, never on total spend. Totals move with traffic, so two versions running at different volumes can't be compared directly. Cost per call holds volume constant, which is what makes the difference attributable to the prompt.
How much can a small prompt edit actually cost?
At 4 million calls a month, 180 extra input tokens is 720 million extra tokens, which is $720.00 a month at a $1.00 per million input rate. The per-call increase is $0.00018 and invisible, which is exactly why it needs measuring rather than noticing.
What if my prompts are versioned in another tool?
That's common and it's only half the problem. The version has to travel on the call and land beside the token counts, otherwise the two systems can't be joined afterwards. Tag at capture, not at analysis.
Can I separate a prompt change from a provider price change?
Only if the version is on the call and rates carry effective dates. With both, a past month recomputes at the rate in force when it ran, so a rate rise and a prompt edit stay two distinct movements rather than one confusing total.
On your infrastructure
Culpa runs on your infrastructure. Your prompts and responses never leave it. Culpa counts calls to run your plan, and it fails open, so if it ever breaks your app keeps running.
How Culpa works
Find the culprit. Not just the total.
Your dashboard shows what you spent. It stops short of who spent it. Culpa shows the conversation, the user and the feature behind it.
Your prompts stay local.
Culpa runs on your own infrastructure. What you send to a model reaches us at no point.
Every dollar has a name.
Follow any charge to the conversation, the user, the feature and the customer behind it.
See the bill before it lands.
Cost your next feature before you ship it. You get the likely bill and the worst case, at best, median, p90 and p99.
Three steps to your first answer.
Change one base URL.
Or drop in the Python or TypeScript library.
Find your most expensive conversation.
In the first session, not the first week.
Cost your next feature before you ship it.
Why the bill went up
Example dashboardCalls traced
418,209
across 3 projects
Spend this week
$378.41
+ $182 vs last week
Failed calls
312
74% retried, and you paid for all of them
+ $182 this week traced to one culprit
Spend over 14 days
Most expensive users
Next week forecast
Graded against reality. Accuracy shown as results land.
Keep reading
Sources: Anthropic pricing. Last reviewed 2026-08-03, rates effective 2026-07-02. Plain text version.