Crabhaus

What it is

The conversion from hardware rental to the per-token prices on an API page. Providers pay for GPUs by the hour; a price per million tokens is that hourly cost spread across every token the machines push through the hour, plus margin. It explains why prices vary 100x across models, why output costs more than input, and where discounts come from.

How it works

Prefill reads the whole prompt in one parallel, compute-bound pass - thousands of input tokens per weight read. Decode emits output one token at a time, streaming the weights and growing KV cache from GPU memory at every step, so generation is bandwidth-bound. Providers batch many requests through the same weight reads; cost per token is roughly hourly cost over sustained tokens per hour.

A concrete example

An 8x H100 node rents for roughly $16-20/hour. If batched serving sustains 1,000 output tokens/second aggregate, that is 3.6M tokens/hour - about $4-6 of raw compute per million output tokens. A provider listing $10-15 per million funds idle capacity, prefill, and margin from the gap; double the throughput through better batching and the cost floor halves.

Why it matters

Agent bills are dominated by the fine print - output multipliers, cache-read discounts, thinking-token billing - not the headline input price. Today's feed is the theory in action: Meta undercuts the frontier with Muse Spark 1.3 and Google sets intro pricing on Gemini 3.8 Flash. Cheaper tokens buy market share; serving efficiency decides who keeps margin.