The context-window cliff: overflow, price tiers and our 402

An agent's context grows one tool result at a time. At some token count, the price of the whole request steps up. At a higher count, the request stops working, and every layer of the stack reports that differently.

Context length is the only request variable that changes both whether a call can succeed and what it costs. Both changes are steps, not slopes. A price computed before the call, as an x402 quote must be, has to know where the steps are.

We checked where they are today. We read the current pricing and error documentation of Anthropic, OpenAI, Google and OpenRouter, and the overflow handling in LiteLLM's source. We ran a self-hosted model past its window to see what comes back. Then we probed our own 402 at both edges. Everything below was read or measured on 2026-10-13, with versions stated.

The short version: our quote is linear across every price tier we tested, and it prices requests that no model in the request can serve. The rest of this post is the evidence, and the fix.

Two cliffs, not one

The first cliff is the price tier. Several providers now charge a higher per-token rate once the prompt crosses a threshold. They apply that rate to the whole request, not only to the tokens above the line.

OpenAI's model pages for the GPT-6 family (gpt-6-astra, gpt-6.1-sol, gpt-6-luna) list a 1,050,000-token context window and the same rule: "Prompts with more than 272K input tokens are priced at 2x input and cache rates and 1.5x output for the full request." The GPT-5.5 page applies 2x and 1.5x "for the full session". The pricing page defines short context as 272K input tokens or fewer.

Google's Gemini pricing draws the line at 200k. Gemini 3.1 Pro Preview costs $2.00 input and $12.00 output per million tokens for prompts up to 200k, and $4.00 and $18.00 above. Gemini 2.5 Pro steps from $1.25/$10.00 to $2.50/$15.00 at the same line. Gemini 3.8 Flash has a flat price, $0.75/$3.75 through December 31, 2026, and a 1,048,576-token input limit.

Anthropic went the other way for most of its line-up. Its pricing page says Claude 4.6 and later models, except Claude Haiku 5.5, bill the full 1M window at standard rates: "A 900k-token request is billed at the same per-token rate as a 9k-token request."

Haiku 5.5 is the exception, and the steepest step we found. Prompts over 100,000 tokens pay $0.50 input and $2.50 output per million, instead of $0.10 and $0.50. That is five times both. The threshold counts all input tokens, including cache reads and cache writes, and "a request over the threshold pays the higher prices even when part of its prompt is a cache hit."

The arithmetic is what makes this a cliff. On GPT-5.5 with 4,000 output tokens, 272,000 input tokens list at $1.48. At 273,000 they list at $2.91. One thousand extra tokens cost $1.43. On Haiku 5.5, the 100,001st prompt token multiplies the whole bill by five.

The second cliff is the window itself, and it is not one number per model either. GPT-5 has a 400,000-token context window but a maximum input of 272,000. GPT-4o has 128,000. Gemini publishes separate input and output limits. Anthropic's context window page gives 1M to Claude Opus 5.5, Sonnet 5.5, Haiku 5.5 and others, and 200k to older models such as the deprecated Sonnet 4.5.

Both thresholds are counted in the provider's tokens. Nobody outside the provider can count those exactly for every model, which is the problem we measured in our tokenizer drift audit. Anthropic's own pricing page notes that Claude 4.7 and later models produce approximately 30% more tokens for the same text than earlier ones. A 100,000-token threshold is a different number of characters on each side of that change.

The window cliff speaks four dialects

When a request does not fit, the provider has to say so. Providers say it in different shapes, and some do not say it as an error at all.

Anthropic distinguishes two cases. If the input alone exceeds the window, the API returns a 400 invalid_request_error ("prompt is too long") on every model. If the input fits but input plus max_tokens does not, Claude 4.5 models and newer accept the request. They stop generating when the window fills, with stop_reason: "model_context_window_exceeded".

That second case is an HTTP 200 carrying a truncated answer. The stop reasons page tells clients to "treat the response as truncated." Separately, requests over 32 MB get a 413 request_too_large. Per the errors page, on the direct API, Cloudflare returns that one before the request reaches Anthropic's servers.

OpenAI's Responses API had an explicit switch. In the published OpenAPI spec (commit 77618e6, 2026-10-13), truncation: "disabled" is the default and makes an oversized input "fail with a 400 error". With "auto", the model will "truncate the response to fit the context window by dropping items from the beginning of the conversation."

The same spec now marks truncation as deprecated. It also carries a context_management array whose only entry type is compaction, triggered at a compact_threshold of at least 1,000 tokens. The error enum for agent session turns has a dedicated value, context_length_exceeded: "The request exceeds the model's context window."

OpenRouter normalizes all of this into its own vocabulary. Its error reference defines a canonical error_type of context_length_exceeded, HTTP 400, for when "the combined input and output tokens exceed the model's context window."

Two details matter for anyone routing through it. In its Responses API skin, that type collapses to the native code invalid_prompt, alongside refusals and generic invalid requests, a point we flagged in the Open Responses audit. And the same section lists context_length_exceeded among errors "transformed into successful completions", with finish reason length.

OpenRouter also ships a fix that is really a policy decision. Its context compression plugin removes or truncates messages "from the middle of the prompt, until the prompt fits within the model's context window." It is opt-in, except that every endpoint with 8,192 tokens of context or less "will default to using context compression."

Its model fallback docs add that any error can trigger the next model, explicitly including "context length validation errors". The request is "priced using the model that was ultimately used."

The fourth dialect is silence. That one we had to measure.

A self-hosted model, past its window

Self-hosted backends are a common last link in a fallback chain: cheap, private, always up. We ran the most common one past its window.

The setup was Ollama 0.30.6 on our own machine, serving qwen2.5:1.5b with num_ctx 2048 and temperature 0, called through its OpenAI-compatible /v1/chat/completions. Each prompt held a short fact ("the secret code is 7341"), about 5,100 tokens of RFC 9110 text, and a question asking for the code. We moved the fact between three positions and ran each five times. A 569-token control prompt answered correctly.

# Ollama 0.30.6, num_ctx 2048, prompts of 5,146-5,165 tokens, n=5 each
placement            HTTP  finish_reason  usage.prompt_tokens  answered 7341
fact at start        200   stop           2047                 0/5
fact at end          200   stop           2047                 5/5
fact in system msg   200   stop           2047                 0/5

Every call returned HTTP 200 with finish_reason: "stop". Nothing in the response said the input had been cut. Only the server log did:

level=WARN msg="truncating input prompt" limit=2048 prompt=5146 keep=4 new=2047

keep=4 is the detail that matters. Ollama kept the first four tokens and the tail, and discarded everything in between. The system message was in the discarded part, so the model answered without it, five times out of five. For an agent, the system prompt is where the task, the tool policy and the spending rules live.

The usage block is quietly wrong for billing, too. It reports the 2,047 tokens the model saw, not the 5,146 that were sent. A gateway that bills on reported usage charges for the truncated request, and gets no signal that it was truncated.

Ollama's API documentation does describe a truncate flag that "returns error if false and context length is exceeded". It is documented only for the embeddings endpoint.

How routers detect overflow: by reading English

A gateway that wants to fall back on overflow first has to recognize it. We read how LiteLLM does it, at commit a5d0535 (2026-10-12), because its exception mapper is effectively a census of how providers phrase the failure.

The generic check, is_error_str_context_window_exceeded in exception_mapping_utils.py, lowercases the provider's error string and looks for nine substrings. They include "this model's maximum context length is", "exceeds the available context size" (commented as llama.cpp) and "exceeds the maximum number of tokens allowed" (commented as Gemini).

Two more patterns catch Cerebras ("current length is" with "while limit is") and "maximum input length is" with "tokens". Provider-specific branches add "prompt is too long", "Input is too long" and "Too many input tokens". In total, the file raises ContextWindowExceededError from 16 places.

One comment in that file shows how far the dependency on phrasing goes. For Bedrock's Mantle endpoint, LiteLLM rewrites overflow errors into the string "prompt is too long", because "Clients such as Claude Code only treat 'prompt is too long' as overflow." The error message has become the API.

Once overflow is detected, LiteLLM's router handles it separately. context_window_fallbacks is its own list. When it is not set, the router logs that it is "Defaulting to fallbacks, if available". Order-based fallbacks are explicitly skipped for this error type.

The router can also avoid the overflow entirely. With enable_pre_call_checks, which defaults to False, it drops any deployment whose max_input_tokens is below the request's token count. That count is taken once per request, by calling token_counter without a model name, which resolves to tiktoken's cl100k_base. A Claude deployment and a Gemini deployment are filtered with the same GPT-4-era count.

The quieter finding is on the success path. LiteLLM translates Anthropic's stop_reason into an OpenAI finish_reason through a fixed map in core_helpers.py. The map has entries for end_turn, max_tokens, tool_use, refusal and even compaction.

It has none for model_context_window_exceeded. Unmapped values log a warning and default to "stop". A Claude answer cut off by the window reaches an OpenAI-shaped client looking finished.

Put the four dialects together and a chain has three ways to learn that it overflowed. It gets a 400 whose English it must parse, a 200 whose finish reason may have been rewritten to stop or length, or nothing at all. Only the first reliably triggers a fallback.

Our own 402, at both edges

Our walk-up quote is public by design. An unauthenticated POST /v1/chat/completions returns a 402 whose payment-required header carries the amount. We used that to read the quote at both cliffs on 2026-10-13.

The input estimate behind the quote is still the one we documented on October 6: ceil(JSON.stringify(messages).length / 4), multiplied by one flat rate per model.

For OpenAI models we can count tokens exactly with a public tokenizer: tiktoken 0.14.0 maps gpt-5.5 to o200k_base. We built payloads from the text of RFC 9110, 9111 and 9112, cut at exact o200k counts, and asked for 4,000 output tokens. The list column applies OpenAI's published short and long rates for GPT-5.5.

# openai/gpt-5.5, max_tokens 4000, o200k counts, 2026-10-13
real tokens  our estimate  our quote   list price  quote/list
    250,000       277,688      $1.82       $1.37        1.33
    272,000       300,716      $1.95       $1.48        1.32
    273,000       301,705      $1.96       $2.91        0.67
    300,000       328,896      $2.12       $3.18        0.67
    500,000       547,538      $3.43       $5.18        0.66
    900,000       990,210      $6.09       $9.18        0.66
  1,100,000     1,206,082      $7.39  over window          -

Below 272,000 tokens, the quote sits above list, as it should. Our estimate runs about 10% high on this text, and the quote includes our margin. One thousand tokens later, the quote sits a third below list, and it stays there to the end of the window. Past the window, at 1.1 million tokens, it still returns a normal 402 for a request GPT-5.5 cannot accept.

For models whose tokenizer is not public, we read the quote at estimated-token pairs on either side of the published threshold, with max_tokens set to 1. The list column assumes the provider counts the same number of tokens our estimate does. It illustrates the step; it is not a bill.

# quote vs list, assuming provider tokens = our estimate, max_tokens 1
model                       tier   est. tokens  our quote     list
anthropic/claude-haiku-5.5  100k        90,008      $0.02  $0.0090
                                       110,008      $0.02  $0.0550
                                       900,008      $0.11  $0.4500
google/gemini-2.5-pro       200k       190,008      $0.29  $0.2375
                                       210,008      $0.32  $0.5250
                                       900,008      $1.36  $2.2500
openai/gpt-6-astra          272k       270,008      $3.25  $2.7001
                                       275,008      $3.31  $5.5002

Across each threshold, the quote moved by zero to six cents. List grew 6.1x between the two Haiku 5.5 points (22% more tokens at five times the rate), 2.2x for Gemini 2.5 Pro and 2x for GPT-6 Astra.

Then the window. We sent requests that the requested model cannot serve, each counted with o200k:

# requests larger than the model's limit, max_tokens 1000
model                      o200k tokens  limit                      our quote
openai/gpt-4o                   150,000  128,000 context window         $0.51
openai/gpt-5                    300,000  272,000 max input              $0.51
openai/gpt-5.5                1,100,000  1,050,000 context window       $7.28
anthropic/claude-opus-5.5     2,500,000  1M context window             $13.20

All four came back as ordinary 402s. The last one is an 11 MB body, under Anthropic's 32 MB request limit, so upstream it would fail as "prompt is too long" rather than a 413. o200k is not Claude's tokenizer, but the margin is 2.5x. In our tokenizer audit, English text varied far less than that across six tokenizers.

Finally, the fallback chain. Our models array, described in the fallback chains post, quotes the most expensive model in the chain. We sent the 300,000-token payload with models: ["openai/gpt-4o", "openai/gpt-5.5"] and 4,000 output tokens. The quote was $2.12, GPT-5.5's flat-rate price.

The first link cannot take that request at all. The only model that can serve it is the second, which lists at $3.18 at long-context rates. Sent to openai/gpt-4o alone, the same payload quoted $1.04 for a call that cannot succeed.

Three findings, then. The quote is linear across every price tier we tested. It does not check the context window, so it prices requests that no model in the request can serve. And the chain quote takes the most expensive model at its flat rate, not at the tier the request will actually land in.

Who pays for each miss

The two errors land on different parties.

An over-window quote mostly costs the buyer time. The official x402 Hono middleware, at commit 6c470a8, cancels settlement when the protected handler returns a status of 400 or more, with the reason handler_failed. An upstream "prompt is too long" therefore settles nothing.

The buyer still paid for a signature, a round trip and the upload of a multi-megabyte body, for a request that was never going to run. The exception is Anthropic's partial case. Input that fits, but not together with max_tokens, comes back as a 200, and a 200 settles.

Many of these quotes would never be signed anyway. The official x402 client SDKs ship a default cap of $1 per payment, which we covered in the spend controls audit. The TypeScript client still sets DEFAULT_MAX_AMOUNT_PER_PAYMENT to "$1" today. GPT-5.5's quote crosses $1 well before its tier, so a default-configured buyer never reaches the long-context band on that model. A buyer on Haiku 5.5 or Gemini 2.5 Pro does, for a few cents.

The under-quote above a tier lands on us. In x402, the buyer's signature caps what can be settled, so the quote is the ceiling. Our tokenizer audit traced where the remainder goes on each of our billing paths; on none of them does the buyer pay it.

That makes the long side of every tier the one band where our walk-up price is reliably below cost. It is also the band most likely to grow, because agent contexts only get longer.

What it means for LLM4Agents

The 402 is our price contract. Context length is the one input where that contract is wrong in both directions: an over-quote below each tier, from margin plus estimator inflation, and a third under list above it. Neither error is random. Both are functions of numbers anyone can read from public pages.

The window check is the cheaper fix, and the more embarrassing one to leave undone. Our public API contract already exposes a contextWindow for every model in /api/v1/models. A 402 that asks for payment on a request the model cannot accept spends the buyer's signature, and our upstream call, on a guaranteed failure.

Fallback chains make the problem worse, and could make it better. Overflow is the most predictable fallback trigger there is: it depends only on a token count and a number in the catalog. Treating it as an upstream error to parse means paying a round trip to learn something we knew before the call. Treating it as a routing input means the chain can skip links that cannot fit, and quote the link that will actually serve, at the tier it will actually hit.

There is a product argument as well as a cost one. Long-context work is where agents most need a price before they commit: whole-repository review, log triage, contract comparison. A flat API key cannot tell an agent that its next call costs twice as much per token because its history crossed 272,000 tokens. A 402 can. Done right, the cliff becomes information we sell, not margin we lose.

The silent dialect is a warning for our own roadmap. Any self-hosted or open-weight link we add inherits whatever its server does on overflow. If that is Ollama's default, our agents get answers to prompts with their instructions removed, behind a 200 and a clean finish reason.

Staying on the frontier

Concretely, and in this order.

First, refuse over-window requests before the 402. If the estimated input exceeds the window of every model in the request, return a 400 with context_length_exceeded, the window and the estimate, instead of a payment requirement. Use a safety band so estimator error cannot reject a request that fits. The band can be tight on models with a public tokenizer.

Second, make the quote tier-aware. Store each model's tiers as data: the threshold, which token classes count toward it (Haiku 5.5 counts cache reads and writes), the multipliers for input, cache and output, and the scope, which on every tier we found is the whole request. Price each request at the tier its estimate falls in. The quote is already an upper bound settled down to the real cost, so a tier-aware bound costs buyers nothing when it overshoots.

Third, count exactly only where it matters. Most requests sit far from any threshold, and the character estimate is good enough there. Within a band around a tier or a window, call an exact counter. Anthropic's token counting endpoint is free and separately rate-limited. OpenAI's input token count endpoint returns "the exact count the model will receive", and o200k runs locally for OpenAI models.

Fourth, route chains on context before the call. Drop links whose window cannot hold the request, then quote the most expensive remaining link at its tier. LiteLLM's pre-call check is the right pattern with the wrong counter: one cl100k count applied to every deployment. Count per tokenizer family, or at least per provider.

Fifth, never report a truncated answer as finished. Map model_context_window_exceeded to length, not stop, and pass the native stop reason through in a response header for agents that need it. Do not enable silent compression by default on any link. Do not add an Ollama-style backend to a chain without making overflow an error.

Sixth, tell agents where the cliffs are. Publish each model's tier thresholds next to contextWindow as structured fields in the model catalog. A planner can then split work below 272,000 tokens, instead of discovering the tier after the fact.

Seventh, turn this post's probes into tests. Threshold pairs for every tiered model, an over-window request for every window size we sell, and a chain whose first link cannot fit. A 402 is free to request, so the suite costs nothing to run, and it would have caught all three findings.