Crabhaus

What matters in AI, and why.

Thu 23 Jul · Issue 009 · 5 min

Issue 009 · Thu 23 Jul

Inside today's edition

01BizUS accuses Moonshot of distilling Anthropic's Fable
02BizAnthropic buys up to 2GW of AMD chips
03LabsAlibaba previews a 2.4T-parameter Qwen Max

The one that matters

Washington accuses Moonshot of distilling Anthropic's Fable

The White House OSTP director says Moonshot AI distilled Anthropic's Fable to build Kimi K3, and the Treasury secretary put sanctions on the table for 'distillation attacks'.

Why it matters

If Washington starts treating distillation as sanctionable IP theft, the open-weights ecosystem builders rely on - much of it Chinese - could change shape fast.

Learn one thing

Temperature, top-p, and why the same prompt gives different answers

Hand-drawn diagram: token-probability bars for build, try, make, paint and moon; the same bars sharpened at low temperature and nearly flat at high temperature; sorted bars with a cobalt bracket over the top three labeled 'top-p: keep the smallest set summing past 0.9'; a die captioned 'sample inside this set'; a caption notes the dice roll repeats for every token.View full-size diagram ↗

What it is

The knobs that control how a model turns predictions into a reply. At each step it assigns a probability to every token in its vocabulary; sampling parameters decide how that table becomes one picked token - locked to the favourite at one extreme, happy to gamble at the other.

How it works

Temperature rescales the model's scores before they become probabilities: low values sharpen the distribution, high values flatten it, zero always takes the top token. Top-p keeps only the smallest set of tokens whose probabilities sum past p and samples inside that nucleus, which resizes itself with the model's confidence. The dice are rolled again for every token.

A concrete example

Ask twice for a weekend project idea at temperature 1.0 and you usually get two different answers: the first token already forks ('Build...' vs 'Try...') and each fork compounds. At temperature 0 a coding agent's retries become nearly reproducible, so agent frameworks pin it low while brainstorming tools turn it up.

Why it matters

A lot of 'flaky LLM' debugging is just sampling doing its job. Use low temperature for extraction, classification and tool calls where consistency matters, and higher settings where variety is the point. It also changes evals: single runs at temperature 1.0 mostly measure luck, not prompts.

What people get wrong: That temperature 0 makes a model deterministic and more truthful. GPU batching and floating-point rounding still cause nondeterminism unless the serving stack is batch-invariant, and temperature reshapes confidence, not knowledge - a wrong fact stays wrong at every setting.

Today's top 5

See the full wire feed →

Get Crabhaus

A concise daily briefing on the AI developments worth understanding. Subscribe on the homepage.