# What is forecast accuracy? > Forecast accuracy needs both a median error and whether reality landed inside the band. Either one alone can make a useless forecast look good. URL: https://getculpa.com/what-is-forecast-accuracy Last reviewed: 2026-08-02 ## Answer Forecast accuracy is two measurements taken together: how far the middle estimate sat from what actually happened, and whether reality landed inside the upper band. Either one on its own can flatter a useless forecast. Culpa, a local-first LLM cost, margin, and forecast ledger, forecasts every period and grades it once the period closes, so the record is scored rather than remembered. ## Why this happens An ungraded forecast is a guess with a decimal point, and grading it on one axis is barely better. Score only the middle estimate and a forecast that happens to centre well looks excellent while its band admits anything. Score only the band and a wide enough range is never wrong and never useful. The pair is what carries information: a tight band that reality keeps landing inside means the model understands your spend, and a wide band nobody ever escapes means it has learned to say nothing safely. ## What this usually looks like - Forecasts get made and never compared against what happened. - Accuracy is discussed as a single percentage with no mention of the band. - Your bands are wide enough that reality has never fallen outside one. - Nobody can say whether last quarter's forecasts were any good. ## Common mistakes - Grading a forecast on its middle estimate alone. Why it hurts: It says nothing about the range you actually planned against, which is what bounds your risk. Do instead: Record the error on the centre and whether the outcome fell inside the band, every period. - Widening the band until nothing ever falls outside it. Why it hurts: A forecast that can't be wrong can't inform a decision either. Do instead: Track band width alongside hit rate. A band that never binds is a band that never helped. - Not storing the forecast that was made. Why it hurts: You can't grade what you didn't keep, so the accuracy question becomes unanswerable forever. Do instead: Persist every forecast at the time it's made, then score it when the period closes. ## Self-check - Find last month's forecast. If it wasn't written down, that's the first finding. - Compute the middle estimate's error against the actual as a percentage. - Check whether the actual landed inside the upper band. - Divide the band width by the middle estimate. A ratio above two admits almost anything. - Do all four for the last six periods, and look at the trend rather than any single month. ## Four months graded on both axes (illustrative) Four closed monthly periods, each with a forecast median, an upper band and an actual. Error is the absolute difference from the actual, as a percentage of the actual. Figures are modelled. Month 1: median $240, band $310, actual $252, error 4.76%, inside the band Month 2: median $240, band $310, actual $301, error 20.27%, inside the band Month 3: median $240, band $310, actual $348, error 31.03%, outside the band Month 4: median $300, band $700, actual $310, error 3.23%, inside the band Month 4 has the best error and the widest band at 2.3 times its median Month 4 scores best on the centre and tells you least, because a band more than double the median would have absorbed almost any outcome. Month 3 is the only period that actually failed, and only the band shows it. ## Cost figures Every figure carries its confidence and its source. No figure on this site is provider-reported. - 3.23% to 31.03% — modelled median forecast error across four graded periods [estimated] Source: Each endpoint is the absolute difference between the modelled median and actual, over the actual, from the teardown arithmetic. A range because the four periods are modelled. - 2.3x — modelled band width as a multiple of the median in the period that scored best on error [calculated] Source: $700 divided by $300, from the month 4 figures in the teardown. ## FAQ Q: How should an LLM spend forecast be graded? A: On two numbers. The percentage error between the middle estimate and the actual, and whether the actual fell inside the upper band. Reporting either alone lets a forecast that centred by luck, or one that hedged into uselessness, look good. Q: What's a good median error on LLM spend? A: There's no defensible figure to quote across products, because volatility differs enormously with traffic shape. The useful target is your own trend, plus a band that binds often enough to mean something when reality stays inside it. Q: Why does the band matter as much as the estimate? A: Because the band is what you plan against. A middle estimate tells you the likely case and the band tells you the case you have to survive, so a forecast graded without it says nothing about whether your budget was ever safe. ## Sources - OpenAI API pricing: https://developers.openai.com/api/docs/pricing Run the free Cost Leak Scan: https://app.getculpa.com/scan?source=pseo&slug=what-is-forecast-accuracy&cluster=metric Machine-readable index of every guide: https://getculpa.com/api/pages Human-readable index of every guide: https://getculpa.com/guides Site overview: https://app.getculpa.com/llms.txt Privacy: Culpa runs on your infrastructure. Your prompts and responses never leave it. Culpa counts calls to run your plan, and it fails open, so if it ever breaks your app keeps running.