Crabhaus

What it is

A way to fine-tune a large model without touching most of it. The pretrained weights stay frozen; small trainable matrices are added alongside them, and only those train - typically well under 1% of the parameter count. It is the workhorse of parameter-efficient fine-tuning (PEFT), and the reason fine-tuning now fits on one consumer GPU.

How it works

The weight change a model needs during adaptation has low intrinsic rank, so LoRA learns it as two thin matrices at small rank r instead of a full update. Gradients flow only in that detour: ~10,000x fewer trainable parameters on GPT-3 175B, on-par quality, no extra inference latency - merge it into the base or ship it as a swappable adapter. QLoRA adds a 4-bit frozen base: 65B on one 48GB GPU.

A concrete example

A team wants its local 8B model to write SQL in the company's house style. Instead of shipping a second full copy of the model, they train a rank-16 adapter overnight on one GPU and get a deliverable measured in tens of megabytes. Their server keeps one base model in memory and swaps adapters per request: SQL for the data team, tone for support replies, each versioned and rolled back like config.

Why it matters

Cheap and swappable is the whole point: one frozen base plus N small adapters replaces N full model copies, and modern inference stacks batch many adapters against a single base. It moved fine-tuning from a lab budget to a hobbyist one - the open tuning ecosystem (Hugging Face PEFT, Unsloth, axolotl) is LoRA-first, and most 'fine-tuned' open models are adapters, not retrained weights.