Crabhaus

What it is

Guardrails are enforcement layers outside the model: schema validators, rule checks, and content classifiers that inspect inputs and outputs, and can block, repair, or retry a response before it reaches a user or a tool. Unlike training-time alignment, they are ordinary software the model cannot talk its way past.

How it works

Checks run at three points. Input rails screen prompts for injection and off-policy asks. Generation rails constrain decoding so the model can only emit tokens matching a declared schema. Output rails run schema, rule, and classifier checks on the result; a failure maps to block, rewrite, or re-prompt with the validation error attached so the model corrects itself.

A concrete example

An expense agent emits a JSON claim: constrained decoding guarantees it parses, a schema check enforces amount and currency, a business rule rejects sums over the approval limit, and a classifier screens the free-text memo. NVIDIA's NeMo Guardrails and the Guardrails AI library package these checks as declared flows instead of ad-hoc if-statements.

Why it matters

Once output feeds tools with side effects - payments, emails, shell commands - validation separates a bad string from a bad action. Each layer fails differently: schema checks pass well-formed nonsense, classifiers get jailbroken, retry loops burn tokens. Layering works because the failure modes are mostly uncorrelated.