Guides / llm cost anomaly detection
LLM cost anomaly detection, and why the total hides the anomaly
LLM cost anomaly detection flags spend that departs from its own pattern. Run it on a company total and it inherits a noise floor, because a single customer tripling can be a fraction of a percent of the whole. Culpa, a local-first LLM cost, margin, and forecast ledger, prices every call, so a detector can watch a customer rather than a company.
Why this happens
Anomaly detection on LLM spend is nearly always pointed at one number, and one number carries a detection floor that grows as you do. The arithmetic is unforgiving. A customer moving from $50 to $150 has tripled, which is the single most actionable thing that happened that week, and inside a $50,000 month it lands as a 0.2% movement. No threshold sensible enough to avoid daily false alarms will ever fire on it. Raise the sensitivity and the alert becomes noise. Lower it and the alert becomes decoration. Neither setting is wrong, because the problem isn't the model behind the detector, it's the object being watched. Point the same detector at each customer and the same event is a 200% movement against that customer's own baseline, which fires on any reasonable setting. That's the whole difference, and it gets decided by whether the spend was ever stored per customer.
What this usually looks like
- Alerts fire on the total and nobody can say which account moved.
- You raised the threshold to stop the noise and now nothing fires at all.
- A customer's usage tripled and the monthly total looked normal throughout.
- Every anomaly investigation starts with a manual breakdown that takes hours.
- Growth makes your detector worse, because the floor rises with the total.
Free, no card, no account
Run the free Cost Leak Scan
It shows your most expensive conversation before you install anything.
Mistakes that cost the most
| Mistake | Why it hurts | Do instead |
|---|---|---|
| Running anomaly detection on the company total. | The detection floor scales with the total, so the bigger you get the more it misses. | Run the same detector per customer, per feature and per model, against each one's own baseline. |
| Fixing a noisy alert by raising the threshold. | It trades false alarms for silence rather than for accuracy, and silence tells you nothing. | Change what's being watched before changing how sensitively it's watched. |
| Alerting on absolute dollars only. | A $100 move is trivial for one account and existential for another, and one threshold can't be both. | Alert on relative movement against each object's own history, with an absolute floor to suppress noise on tiny accounts. |
| Detecting without attributing. | An alert that names a number rather than a customer starts an investigation instead of ending one. | Make the alert carry the customer, feature and prompt version that moved. |
Run this check tonight
- Find your current cost alert and write down exactly what number it watches.
- Work out what a single customer would have to do to move that number past its threshold.
- Ask whether any real customer could plausibly do that.
- Check whether your last cost alert named an account or just a total.
The same event, watched two ways
Illustrative example
A modelled month at $50,000 of total spend, priced on Claude Haiku 4.5 at real rates of $1.00 and $5.00 per million from the price book effective 2026-07-02. One customer moves from $50.00 to $150.00. A common alert threshold of 5% is applied to each object. Volumes are modelled and the rates are real.
Nothing about the detector changed between those two lines. The only thing that changed is what it was pointed at, and that choice was made when the data was written rather than when the alert was configured.
Every number, with its confidence and source
| Figure | What it means | Confidence | Source |
|---|---|---|---|
| 0.2% | a modelled customer tripling their spend, expressed as a share of the company total | calculated | A $100.00 movement against a modelled $50,000.00 monthly total = 0.2%, against 200% when measured against that customer's own $50.00 baseline. Spend figures are modelled and priced on Claude Haiku 4.5 at real rates of $1.00 and $5.00 per million from the price book effective 2026-07-02. The pair is the finding rather than either number alone. |
What a generic answer can’t know
A detector can only watch objects that exist in the data. That's why this looks like an alerting problem and is actually a storage one: a platform holding spend per agent can alert per agent and no finer, whichever anomaly model it runs. Culpa prices every call and tags it with the customer, the feature, the prompt version and the run, so the same detector can be pointed at any of them and each gets its own baseline. It also holds the revenue each customer pays, which turns a cost anomaly into the question that matters: a customer whose spend tripled is only a problem if their payment didn't.
Questions founders ask next
Why do cost alerts miss the things that matter?
Because they usually watch a total, and a total has a detection floor proportional to its size. A customer tripling from $50 to $150 is a 0.2% movement inside a $50,000 month. No threshold that avoids daily false alarms will fire on it, and the bigger you grow the worse that gets.
Should I just lower the alert threshold?
That trades silence for noise rather than for accuracy. The fix is to change what's being watched, not how sensitively. The same detector at 5% misses a 0.2% movement on the total and catches the identical event as 200% against that customer's own baseline.
What should an LLM cost alert actually say?
Which object moved, by how much against its own history, and what it costs against what that account pays you. An alert naming a number starts an investigation. An alert naming a customer, a feature and a prompt version ends one.
Can I add per-customer detection to my existing tool?
Only if it already stores spend per customer. Anomaly detection is a read-time operation over whatever was written, so a platform that aggregated at the agent can never be persuaded to alert on a conversation. That's a data decision rather than a configuration one.
On your infrastructure
Culpa runs on your infrastructure. Your prompts and responses never leave it. Culpa counts calls to run your plan, and it fails open, so if it ever breaks your app keeps running.
How Culpa works
Find the culprit. Not just the total.
Your dashboard shows what you spent. It stops short of who spent it. Culpa shows the conversation, the user and the feature behind it.
Your prompts stay local.
Culpa runs on your own infrastructure. What you send to a model reaches us at no point.
Every dollar has a name.
Follow any charge to the conversation, the user, the feature and the customer behind it.
See the bill before it lands.
Cost your next feature before you ship it. You get the likely bill and the worst case, at best, median, p90 and p99.
Three steps to your first answer.
Change one base URL.
Or drop in the Python or TypeScript library.
Find your most expensive conversation.
In the first session, not the first week.
Cost your next feature before you ship it.
Why the bill went up
Example dashboardCalls traced
418,209
across 3 projects
Spend this week
$378.41
+ $182 vs last week
Failed calls
312
74% retried, and you paid for all of them
+ $182 this week traced to one culprit
Spend over 14 days
Most expensive users
Next week forecast
Graded against reality. Accuracy shown as results land.
Keep reading
Sources: Anthropic pricing. Last reviewed 2026-08-03, rates effective 2026-07-02. Plain text version.