Crabhaus

What it is

An asynchronous way to call OpenAI, Anthropic, and Google models: submit a job of many requests at once, the provider processes it within a completion window of up to 24 hours, and every token is billed at roughly half the realtime API price. Same models, same output quality - the only thing you give up is an immediate response.

How it works

You package requests - OpenAI takes a JSONL file, one request per line; Anthropic takes up to 100,000 requests per batch - each tagged with your own ID. The provider schedules the job onto spare capacity between realtime traffic peaks, which is exactly why it can discount. Poll the batch status, then match results back by ID. Jobs often finish in minutes; 24 hours is the ceiling, not the norm.

A concrete example

A nightly eval suite of 5,000 test prompts has no user waiting on it. Run it through the batch endpoint after midnight: same model, same prompts, half the token bill, results ready before morning. The same shape fits backfills - classifying a 2M-row archive - plus bulk summarization and embedding jobs.

Why it matters

It is one of the few cost levers that needs no model downgrade, no prompt surgery, and no quality trade-off. Evals, backfills, classification, and report generation quietly dominate many teams' token bills, and all of them tolerate latency. Batch quotas are also separate from realtime limits, so big offline jobs stop competing with live traffic.