Crabhaus

What it is

Provider-side reuse of the computation behind a repeated prompt prefix. When two API requests start with the same tokens, the provider skips re-processing the shared part and bills it at a steep discount: a cached read on Anthropic's API costs one tenth of the normal input rate. It matters because agent workloads resend nearly identical context on every call.

How it works

Models build an internal attention state per token; caching stores that state keyed to the exact prefix, so a matching request resumes instead of recomputing. Matching is literal, and one changed token invalidates everything after it. Anthropic bills cache writes at 1.25x-2x base and reads at 0.1x; OpenAI caches 1,024+ token prompts automatically; DeepSeek caches every request to disk by default.

A concrete example

A coding agent carries a 12,000-token system prompt and tool set across 40 calls. Uncached on Claude Opus 5 ($5 per million input) that prefix costs about $2.40. Cached, call one writes for roughly 7.5 cents and 39 reads at $0.50 per million add about 23 cents: near $0.31 total, an 87% cut.

Why it matters

Agent loops resend the whole transcript every turn, so input tokens dominate the bill. Putting stable instructions and tools first and volatile content last is one of the highest-leverage cost levers a builder controls, and a big reason long-running agent harnesses cost less than raw token math suggests.