Crabhaus

What it is

A retry re-issues a failed call to the same provider; a fallback reroutes to an alternative - another provider, a smaller model, or a cached answer; graceful degradation ships a reduced but honest result when nothing upstream succeeds. Together they turn routine provider errors - 429s, overloads, timeouts, malformed outputs - into brief slowdowns instead of crashes.

How it works

Only transient failures are worth retrying; auth errors and context overflows fail identically on attempt two. Retries wait exponentially longer with random jitter, capped at 2-3 attempts inside a retry budget or circuit breaker that fails fast when error rates spike. Then a fallback chain takes over - second provider, smaller model, cached response - validating output at each hop.

A concrete example

An agent pipeline summarizing 50 documents starts getting 429s at document 12. Naive code hammers the API and dies. Handled: each call retries twice with jittered backoff, a router falls back to a smaller model for the low-stakes remainder, and two failed summaries are listed as 'not summarized'. The job finishes ninety seconds late instead of not at all.

Why it matters

Provider errors are routine: every major LLM API sheds load at peak, and agent workloads multiply exposure - one flaky hop can sink a 40-step run. Most of the gap between a demo and a production system is what happens on the bad path. Done badly, retries become the outage: synchronized, unjittered retry loops are a classic cause of cascading failure.