Guides / idle llm cost
The AI costs that accrue when you make no calls at all
Some provider charges bill by elapsed time rather than per request, so they accrue while your product sits idle. Held caches, running containers and agent sessions all cost money at zero calls. Culpa, a local-first LLM cost, margin, and forecast ledger, prices calls exactly, and these belong to a different shape worth measuring separately.
Why this happens
Almost everything written about AI cost assumes cost follows usage, and for tokens it does. Three charges break that assumption by billing on elapsed time. A context cache bills for the tokens it holds, every hour it holds them, whether you read it once or never. A container or code-execution sandbox bills for as long as it's alive. An agent session bills for its running time. None of them produce a request, so none of them appear in anything that counts requests, and they're invisible to any tool built on a log of calls. The reason this bites rather than merely annoys is that the cheapest way to make something fast is usually to leave it warm, and warm is exactly what these charges price. A cache with a generous lifetime and a container left running are both rational engineering decisions that quietly convert latency into rent.
What this usually looks like
- Your invoice has line items that aren't token charges and nobody owns them.
- Spend doesn't fall as much as expected during a quiet week.
- A container or sandbox has been running longer than anyone remembers.
- Cache lifetimes were set once and never revisited.
- Your cost tooling reconciles on calls and still comes in under the invoice.
Free, no card, no account
Run the free Cost Leak Scan
It shows your most expensive conversation before you install anything.
Mistakes that cost the most
| Mistake | Why it hurts | Do instead |
|---|---|---|
| Assuming cost follows usage. | Time-based charges accrue at zero usage, so a quiet period costs more than it looks like it should. | Separate charges that follow requests from charges that follow the clock. |
| Setting a generous cache lifetime for safety. | Storage bills for hours held, so an over-long lifetime is rent on context nobody read. | Set the lifetime from how long you actually reuse the context, then check it against reality. |
| Leaving containers warm to avoid cold starts. | It's a real latency win and a standing charge, and only one of the two is usually measured. | Price the warm hours and decide deliberately, rather than defaulting to always-on. |
| Sizing a container generously because it's cheap. | Size and time multiply, so the largest tier costs many times the smallest for identical hours. | Size to the job, and revisit it once you've seen real memory use. |
Run this check tonight
- Find the non-token line items on last month's provider invoice and total them.
- List every cache, container and session you hold, and how long each one lives.
- Work out what a quiet weekend costs you before a single call is made.
- Check whether any resource has been running longer than the work that needed it.
What a quiet month costs at zero calls
Illustrative example
Published rates read 2026-08-05. Gemini charges context cache storage at $1.00 per 1,000,000 tokens per hour on most models, rising to $8.10 on 2.5 Pro Priority. OpenAI charges container sessions at $1.92 per 20-minute session for the 64 GB tier. Anthropic charges code execution at $0.05 per hour per container with 1,550 free hours per organisation per month. The holdings below are modelled, every rate is published.
Every figure above is what you pay having made no requests at all. The Anthropic number is the encouraging one and it's worth stating plainly, because a free allowance that generous means most teams owe nothing there. The Gemini and OpenAI numbers are the ones to go and check tonight.
Every number, with its confidence and source
| Figure | What it means | Confidence | Source |
|---|---|---|---|
| $144.00 per month | modelled cost of holding a 200,000-token Gemini context cache, making no calls | calculated | Gemini's pricing page, read 2026-08-05, publishes context caching storage at $1.00 per 1,000,000 tokens per hour on most models. 0.2M tokens x $1.00 x 24 hours x 30 days = $144.00. On the 2.5 Pro Priority rate of $8.10 the same holding is $1,166.40. The rates are published, the cache size and the holding period are modelled. |
| $4,147.20 per month | modelled cost of one always-on 64 GB OpenAI container, making no calls | calculated | OpenAI's pricing page, read 2026-08-05, publishes container sessions at $1.92 per 20-minute session for the 64 GB tier, billed by the minute with a five-minute minimum, so $5.76 an hour, so $5.76 x 24 x 30 = $4,147.20. The rate and the tier are published, the always-on assumption is modelled. |
What a generic answer can’t know
This is a shape rather than a gap, and being straight about it matters more than a sales line. Culpa prices calls, exactly and in decimal, from a versioned price book. Time-based charges are a different object: they belong to a resource you hold rather than a call you made, so recording them needs the resource tagged with its start, its end and its size, and that record has to be written by whatever creates the resource. Nothing that watches traffic can reconstruct it afterwards, which is why call-based tools miss these across the board rather than as an oversight. The practical answer today is to take the non-token lines from your provider console, add them to your call-based total, and keep the two separated so you can see which one is growing.
Questions founders ask next
Can an LLM cost money when nobody is using it?
Yes. A held Gemini context cache bills for the tokens it stores every hour it stores them, so a 200,000-token cache costs $144.00 a month at the standard rate with zero calls. Running containers and agent sessions bill the same way, on elapsed time rather than requests.
Why doesn't my cost tool show these charges?
Because they produce no request. Cost tooling almost universally works from a log of calls, and a cache sitting idle or a container sitting warm generates no call to log. It isn't a capture bug so much as a shape the call-based model has no room for.
How do I find my exposure?
Open your provider console and look at the line items that aren't token charges. Gemini itemises caching storage, OpenAI itemises container and training charges, Anthropic itemises code execution. Compare that total against the gap between your invoice and whatever your cost tooling reports.
Is keeping a cache warm still worth it?
Often yes, and it's a trade rather than a mistake. Cache reads cost a fraction of fresh input tokens, so a cache read enough times pays for its storage several times over. The failure mode is a long lifetime on context nobody reads, which is rent with no return. Price the hours against the reads.
On your infrastructure
Culpa runs on your infrastructure. Your prompts and responses never leave it. Culpa counts calls to run your plan, and it fails open, so if it ever breaks your app keeps running.
How Culpa works
Find the culprit. Not just the total.
Your dashboard shows what you spent. It stops short of who spent it. Culpa shows the conversation, the user and the feature behind it.
Your prompts stay local.
Culpa runs on your own infrastructure. What you send to a model reaches us at no point.
Every dollar has a name.
Follow any charge to the conversation, the user, the feature and the customer behind it.
See the bill before it lands.
Cost your next feature before you ship it. You get the likely bill and the worst case, at best, median, p90 and p99.
Three steps to your first answer.
Change one base URL.
Or drop in the Python or TypeScript library.
Find your most expensive conversation.
In the first session, not the first week.
Cost your next feature before you ship it.
Why the bill went up
Example dashboardCalls traced
418,209
across 3 projects
Spend this week
$378.41
+ $182 vs last week
Failed calls
312
74% retried, and you paid for all of them
+ $182 this week traced to one culprit
Spend over 14 days
Most expensive users
Next week forecast
Graded against reality. Accuracy shown as results land.
Keep reading
Sources: Gemini API pricing, OpenAI API pricing, Anthropic pricing. Last reviewed 2026-08-05, rates effective 2026-07-02. Plain text version.