Crabhaus

What it is

Structured output is getting an LLM to return machine-parseable data — usually JSON matching your schema — instead of prose. Constrained decoding is the strong enforcement: the inference engine restricts every sampling step so only tokens that keep the output valid can be chosen.

How it works

The schema or grammar is compiled into an automaton over the token vocabulary. At each step the engine masks the logits of every token that would break the structure and samples from the rest — the model picks the values, but the syntax can't derail. Willard & Louf showed the masking can be precomputed cheaply; Outlines, XGrammar, and API schema modes build on it.

A concrete example

An agent extracts {vendor, date, total} from invoices. Prompt-only 'reply in JSON' still yields the odd trailing comma or missing field, so you write retry logic. Pass the schema to a structured-output API and every response parses with all three keys — you only check whether the values are right.

Why it matters

Every tool call an agent makes is structured output; one malformed argument object means a failed step and a retry. Enforcement in the sampler deletes a class of defensive code and makes cost and latency predictable — no re-asking the model until it complies.