Reducing LLM spend in the order that actually works

When a bill needs to come down, teams usually start with prompt caching, because it's technical, safe and doesn't change any answers. It's a real saving and it's rarely the largest one available. Model choice usually dominates everything else by a wide margin, because the rate spread between model tiers is much larger than any efficiency you can win at a fixed tier. The reason caching gets picked first is that it costs nothing politically: nobody has to agree that quality can drop. Model choice needs that conversation, so it goes last, which is exactly backwards from the order the numbers suggest. The right sequence is to measure what each lever is worth on your own traffic before touching any of them, then start with the largest that your quality bar can absorb. Two cautions worth stating. Some levers stack and some don't, and caching a prompt you're about to shorten is wasted work. And the fastest saving of all is usually removing a call nobody needed, which shows up in none of the lever lists because it isn't an optimisation.

• The cost-reduction plan starts with caching and never reaches model choice.
• Nobody has priced what each lever is worth on your own traffic.
• Quality is treated as fixed, so the largest lever is off the table by default.
• Savings were implemented in the order they were easiest to ship.
• A round of optimisation cut less than expected and nobody sized it first.

1. Re-price last month's traffic on the next model tier down.
2. Work out what share of your input is a stable prefix worth caching.
3. Find your retry rate and price the retries separately.
4. Look for calls whose output nothing reads, and price those on their own.

Illustrative example

A modelled million calls a month at 2,000 input and 500 output tokens, running on Claude Opus 5 at its real rates of $5.00 and $25.00 per million. Lever one moves the workload to Claude Haiku 4.5 at $1.00 and $5.00. Lever two keeps Opus 5 and caches 1,500 of the 2,000 input tokens, read at the cached rate of $0.50 per million. Rates are published, the volume, token shape and cacheable share are modelled.

Eighty points against thirty, on the same month, and the thirty is the one that gets shipped first because it needs nobody's permission. Neither number is a recommendation on its own: the model move is only available if the cheaper model holds up on your cases. Measuring both before choosing is the entire point, and it takes an afternoon.

Every lever's value depends on your own traffic. What share of your input is a stable prefix, how many retries you run, how much of your output nothing reads, and whether a cheaper model holds up on your cases are all facts about you, and none appear on a pricing page. Culpa prices every call from a versioned price book in exact decimal and keeps the token shapes alongside the costs, so each lever can be sized against last month's real traffic before anyone writes code. It also carries whatever identifiers you attach, which is what turns the ordering from a general list into a specific one: which feature would benefit most, which customers sit behind the expensive calls, and whether the saving you're about to make is worth the quality risk on the traffic that actually produces revenue.

• LLM Pricing Modifiers: Which Ones Stack and Which Ones Conflict
• Local-First LLM Observability: Table Stakes, Not a Differentiator
• LLM Cost in Grafana: Per-Agent Dollars, and a Token Bill of Its Own
• LLM Cost in New Relic: Your Traces Bill by the Gigabyte