# Context window cost optimization

> One token past Gemini's 200k boundary doubles the input rate and lifts the output rate by half. The whole prompt reprices, not the excess.

URL: https://getculpa.com/context-window-cost-optimization
Last reviewed: 2026-08-02

## Answer
Gemini charges $1.25 per million input on prompts up to 200k tokens and $2.50 above it, and the output rate moves with the prompt too, from $10.00 to $15.00. So one token past the boundary reprices the entire call rather than the excess. Culpa, a local-first LLM cost, margin, and forecast ledger, measures how many of your prompts sit just over it.

## Why this happens
Almost every piece of advice about prompt size assumes a slope, where trimming tokens saves money in proportion. Tiered context pricing is a step. Gemini 2.5 Pro and 3.1 Pro both publish two rate cards split at 200k tokens, and crossing that line doubles the input rate on the whole prompt. The part that surprises people is the output column, which also moves, so a long prompt raises the price of tokens the prompt never touched. A retrieval change that adds one percent of context can add most of a bill, and an average prompt size tells you nothing about it, because the cost depends on the share of calls sitting the wrong side of one number.

## What this usually looks like
- Spend jumped after a retrieval change that added a small amount of context.
- Your average prompt size looks fine and your bill doesn't.
- Nobody can say what share of your calls exceed 200,000 tokens.
- Output cost rose while your responses stayed the same length.

## Common mistakes
- Treating prompt size as a smooth cost curve. Why it hurts: The boundary is a step, so the last token before it costs far more than the first. Do instead: Find the boundary for your model and measure the distribution of prompts around it.
- Reasoning about average prompt size. Why it hurts: An average sitting under the boundary hides a tail that crosses it on every expensive call. Do instead: Work in percentiles. The share of calls above the line is the number that bills you.
- Assuming only input reprices. Why it hurts: Gemini's output rate is set by prompt size, so long prompts tax your responses as well. Do instead: Price both columns at both tiers before deciding a context change is cheap.

## Self-check
- Find whether your model publishes a context tier and what token boundary splits it.
- Plot your prompt sizes as a distribution rather than an average.
- Establish what share of your calls land above the boundary.
- Price a month at both tiers and treat the gap as your exposure to context growth.
- Re-check after every retrieval or memory change, because those move prompt size directly.

## One percent more context, ninety-eight percent more cost (illustrative)
Gemini 2.5 Pro, whose published rates split at 200,000 tokens: $1.25 per million input and $10.00 output at or below the boundary, $2.50 and $15.00 above it, read from Gemini's pricing page 2026-08-02. Culpa's price book carries only the lower tier. 100,000 calls a month each returning 2,000 output tokens. Volumes are modelled.
  Prompts at 199,000 tokens: (199,000 x $1.25 + 2,000 x $10.00) per million = $0.24875 + $0.02 = $0.26875
    Monthly: 100,000 x $0.26875 = $26,875
  Prompts at 201,000 tokens: (201,000 x $2.50 + 2,000 x $15.00) per million = $0.5025 + $0.03 = $0.5325
    Monthly: 100,000 x $0.5325 = $53,250
  One percent more context costs $26,375 a month more, a 98% increase
  At the boundary itself: 200,000 tokens bills $0.25 of input and 200,001 bills $0.50
A single token doubles the input cost of a call, and the output rate rises by half on tokens the prompt never touched. This is the one place where trimming context is worth vastly more than it looks.

## Cost figures
Every figure carries its confidence and its source. No figure on this site is provider-reported.
- 98% — modelled monthly cost increase from raising prompts one percent across the 200k boundary [calculated] Source: $53,250 divided by $26,875, from the teardown arithmetic at Gemini 2.5 Pro's two published tiers.
- $26,875 to $53,250 — modelled monthly cost of the same workload either side of the boundary [estimated] Source: Both endpoints from the teardown arithmetic at rates read off Gemini's pricing page 2026-08-02. The above-200k tier isn't in Culpa's price book. A range because the call volumes are modelled.

## FAQ
Q: Does the higher rate apply to the whole prompt or only the excess?
A: The whole prompt. Gemini publishes the rate as a price for prompts above 200k tokens rather than as a surcharge on the tokens past it, so a 201,000-token prompt prices all 201,000 at the higher rate.

Q: Which models have a context tier?
A: On Gemini it's a Pro-model property, checked 2026-08-02: 2.5 Pro and 3.1 Pro split at 200k while 2.5 Flash, 3.5 Flash and 3.1 Flash-Lite publish one rate. OpenAI's GPT-5.6 family publishes short and long context columns without stating the boundary in the table.

Q: Why did my output cost rise when responses stayed the same length?
A: Because the output rate is selected by prompt size on a tiered model. Crossing the boundary lifts Gemini 2.5 Pro's output from $10.00 to $15.00 per million, so identical responses bill 50% more on the strength of a longer prompt.

## Sources
- Gemini API pricing: https://ai.google.dev/gemini-api/docs/pricing
- OpenAI API pricing: https://developers.openai.com/api/docs/pricing

Run the free Cost Leak Scan: https://app.getculpa.com/scan?source=pseo&slug=context-window-cost-optimization&cluster=optimization

Machine-readable index of every guide: https://getculpa.com/api/pages
Human-readable index of every guide: https://getculpa.com/guides
Site overview: https://app.getculpa.com/llms.txt

Privacy: Culpa runs on your infrastructure. Your prompts and responses never leave it. Culpa counts calls to run your plan, and it fails open, so if it ever breaks your app keeps running.