Crabhaus

What matters in AI, and why.

Sun 4 Oct · Issue 082 · 3 min By Vatsal Mishra

Issue 082 · Sun 4 Oct

Kolibri lands; agents graded on what they leave

01LabsAleph Alpha's Kolibri: open 78B MoE, 1M context
02ResearchMicrosoft grades agents on backend state, not prose
03BizAnother OpenAI safety leader quits, warns publicly

The one that matters

Aleph Alpha's Kolibri lands: a sovereign open-weight MoE with a 1M-token context

Released on German Reunification Day, Kolibri is an English-German Mixture-of-Experts model with 78B total parameters, 3B active, context up to 1M tokens, Apache 2.0, weights on Hugging Face. Aleph Alpha credits a training pipeline engineered to run pre-training without a person stepping in when hardware failed.

Why it matters

Europe's 'sovereign AI' push has mostly produced announcements; this is an actual artifact: permissive license, long context, and a 3B-active footprint cheap enough to self-host. For builders it is a serious English-German option, and a signal that mid-size labs can ship competitive open models off a well-engineered pipeline rather than raw scale.

Learn one thing

Synthetic training data

Hand-drawn diagram titled 'Synthetic training data': seeds become prompts, a generator model writes raw synthetic text, a funnel applies dedup, quality scoring and decontamination, and the output joins human data in a training mix that trains a student model. An inset, 'model collapse (recursive loop)', shows training on synthetic data alone as distributions lose their tails.View full-size diagram ↗

What it is

Training data a model generates instead of data collected from people. It covers teacher-to-student examples (distillation, instruction data) and pretraining-scale corpora of model-written text. Hugging Face's Cosmopedia is the open reference: 30M+ files, 25B tokens, generated with Mixtral to reproduce Microsoft's Phi-1.5 recipe.

How it works

Pipelines start from seeds — curated topics, web excerpts, a syllabus — turned into prompts that fix subject, audience, and style. The generator writes; filters do the real work: dedup, quality scoring, and decontamination against benchmarks. Success hinges on seed diversity, so outputs don't cluster, and on mixing synthetic with human data rather than replacing it.

A concrete example

Phi-1 trained a 1.3B model on filtered 'textbook quality' data, partly GPT-3.5-generated, and hit 50.6% pass@1 on HumanEval, competitive with far larger code models. Cosmopedia rebuilt the playbook openly; its authors found the hard problem wasn't generation but keeping 30 million prompts diverse.

Why it matters

It breaks the annotation bottleneck: instruction datasets that needed armies of human labelers now come from a teacher model in days. Labs get targeted coverage of rare skills, formats, and languages, and small open models inherit frontier behavior through distillation — though licenses differ on whether training on outputs is allowed.

What people get wrong: That 'model collapse' proves synthetic data poisons every model it touches. The Nature 2024 result concerns recursive loops where models train mostly on predecessors' output and real data vanishes — distribution tails go first. Curated pipelines keep human data in the mix, filter hard, and seed for diversity: a different regime.

See every explainer, in order →

Today's top 5

ResearchMicrosoft and HF's ThinkingBox grades agents on the backend state they leave behind, re-running tasks to score reliability, not prose

Benchmarks that grade final answers miss agents that say 'done' while the database disagrees. Scoring the terminal state of real tool backends, and re-running the same task twenty times to measure consistency, is a sharper reliability bar. Expect repeated-run pass rates to start showing up in agent marketing.

BizAn OpenAI safety leader quit, calling the company's culture 'broken'; he wrote the safety reports for its major model releases

The critique comes from the person who wrote the safety reports attached to major releases, not an outside activist. Safety-team departures with public warnings are now a recurring pattern at OpenAI, and each one hands regulators ammunition exactly as the company pushes always-on agents into enterprises.

AgentsSimon Willison: pay-per-use AI APIs need default hard budget caps; warning emails won't cut it once coding agents spend on your behalf

Agents turn API spend from a human-paced decision into an unattended loop, and one bad prompt can burn a month's budget overnight. The point is product design, not thrift: hard caps that return errors beat warning emails nobody reads at 3am. Vendors that ship real caps will win the agent-hosting trust fight.

Agentsdatalab's Lift, a 9B open-weights model, extracts schema-matching JSON from PDFs in one pass and returns null instead of guessing

Document extraction is one of the most common production LLM jobs, and it usually means OCR plus an LLM plus JSON-repair glue. A single 9B open-weights model that reads whole documents and returns schema-valid JSON, with nulls instead of hallucinated values, collapses that stack into something self-hostable.

BizA US federal judge ruled a warrantless Flock license-plate search unconstitutional: 'indiscriminate mass surveillance'

One of the first federal rulings to call a warrantless Flock lookup unconstitutional, even though it sets no binding precedent. AI-powered camera networks were sold to police as too useful to question; courts are starting to treat querying them like the search it is. Expect this citation in suppression motions.

See the full wire feed →

Quick check

Was this issue useful?

Get Crabhaus in your inbox

A concise daily briefing on the AI developments worth understanding.