Guides / track llm cost openai
How to track LLM cost on OpenAI
A streamed OpenAI response carries token usage only when the request asks for it, so a streaming app that leaves that setting off records calls with no numbers attached. Culpa, a local-first LLM cost, margin, and forecast ledger, counts what arrives and marks a call with no usage as a floor rather than guessing at it, so the gap shows up as a flag instead of as a quiet zero.
Why this happens
A non-streamed response carries a usage object and everything downstream works. A streamed one carries usage in a final event that the provider sends only when the request opted in. Miss that opt-in and the stream ends with content and no counts, which leaves any ledger with a call it can identify and can't price. The failure is quiet in the sense that nothing errors, and loud in the sense that the calls arrive flagged, so the tell is sitting in your own data waiting for somebody to sort by it.
What this usually looks like
- A large share of your calls carry no token counts at all rather than low ones.
- Your streamed features are the ones missing numbers and your non-streamed features aren't.
- Ledger totals sit far below the invoice while the call count looks right.
- Nobody has sorted the ledger by the completeness flag, so the pattern has never surfaced.
Free, no card, no account
Run the free Cost Leak Scan
It shows your most expensive conversation before you install anything.
Mistakes that cost the most
| Mistake | Why it hurts | Do instead |
|---|---|---|
| Assuming a streamed response carries usage the way a normal one does. | It doesn't unless the request opted in, so the counts you need never arrive. | Turn the usage option on for every streaming call and confirm the counts land. |
| Reading a zero-cost call as a cheap call. | A zero with a completeness flag means unpriceable, and it can sit on top of real spend. | Split your ledger by the flag before reading any total. Flagged rows are floors, not facts. |
| Comparing cached and total input without knowing which contains which. | Prompt tokens already include the cached ones, so counting both double-counts the prefix. | Take input as prompt minus cached, which is what the ledger stores and what you pay full rate on. |
Run this check tonight
- Count calls with no token usage at all, then check which of them streamed.
- Confirm your streaming requests ask for the usage event, and check one response to be sure.
- Sort last month by the completeness flag and total the flagged rows separately.
- Compare the unflagged total against the invoice. The rest is what the flags are hiding.
- Check that your own arithmetic takes input as prompt minus cached rather than adding the two.
Three million streamed calls with the usage option off
Illustrative example
A streaming product making 3 million calls a month on GPT-5.4 at real rates of $0.0025 per 1k input and $0.015 output, each sending 3,000 input tokens and returning 500. Volumes are modelled.
The ledger was never wrong, it was empty and said so. Three million flagged rows is the loudest signal in the system, and the fix is a single setting on the request.
Every number, with its confidence and source
| Figure | What it means | Confidence | Source |
|---|---|---|---|
| $45,000 | modelled monthly spend that goes unpriced when streamed calls carry no usage event | calculated | 3,000,000 calls x $0.015, using real GPT-5.4 rates per 1k from the price book, effective 2026-07-02. Call volume and token counts are modelled. |
| $0.00 | what the ledger records for a call with no usage, alongside an incomplete flag | calculated | An absent usage field is stored as missing rather than guessed, so the priced total for that call is zero and the completeness flag carries the meaning. |
What a generic answer can’t know
A provider knows what it billed and has no view of what your ledger managed to record. The gap between those two numbers, and which of your features sits inside it, exists only where both are held together. Culpa keeps that on your infrastructure, where your prompts and responses stay, and counts the calls to run your plan.
Questions founders ask next
Why do my streamed OpenAI calls have no token counts?
Because a streamed response carries usage only in a final event the provider sends when the request opted in. Without that, the stream delivers content and no counts, so any ledger downstream has a call it can name and can't price.
What does Culpa do with a call it can't price?
It stores the call at zero cost and marks it incomplete rather than estimating. The reporting layer then reads that flag as an estimate rather than a calculation, so a floor never gets presented as a measurement.
Do cached tokens count twice?
No. Prompt tokens already include the cached ones, so the portion billable at the full input rate is prompt minus cached. That's the subtraction the ledger stores, and doing it the other way inflates your input volume by the size of your cached prefix.
On your infrastructure
Culpa runs on your infrastructure. Your prompts and responses never leave it. Culpa counts calls to run your plan, and it fails open, so if it ever breaks your app keeps running.
How Culpa works
Find the culprit. Not just the total.
Your dashboard shows what you spent. It stops short of who spent it. Culpa shows the conversation, the user and the feature behind it.
Your prompts stay local.
Culpa runs on your own infrastructure. What you send to a model reaches us at no point.
Every dollar has a name.
Follow any charge to the conversation, the user, the feature and the customer behind it.
See the bill before it lands.
Cost your next feature before you ship it. You get the likely bill and the worst case, at best, median, p90 and p99.
Three steps to your first answer.
Change one base URL.
Or drop in the Python or TypeScript library.
Find your most expensive conversation.
In the first session, not the first week.
Cost your next feature before you ship it.
Why the bill went up
Example dashboardCalls traced
418,209
across 3 projects
Spend this week
$378.41
+ $182 vs last week
Failed calls
312
74% retried, and you paid for all of them
+ $182 this week traced to one culprit
Spend over 14 days
Most expensive users
Next week forecast
Graded against reality. Accuracy shown as results land.
Free, no card, no account
Run the free Cost Leak Scan
It shows your most expensive conversation before you install anything.
Keep reading
Sources: OpenAI API pricing. Last reviewed 2026-08-02. Plain text version.