Idrees Patel

Last verified: 7 November 2026. Government and company claims are attributed throughout. None has been tested in court.

A support team trains a small, cheap model to answer routine questions by learning from its own large model. A lab allegedly runs thousands of fake accounts to harvest a rival’s reasoning, then trains on the results. Both are model distillation. The technique is the same. What separates them is permission.

That gap is now a policy fight. In September 2026, three US agencies accused six Chinese AI companies of “industrial-scale” distillation of American models, and China rejected the claims. In October, Anthropic said new safeguards had cut the data distillers harvest from its platform by 90 to 99%.

Key Takeaways

• Model distillation is a 2015 technique built for efficiency. Hinton, Vinyals and Dean used soft targets to compress large models into small ones.
• A student learns behaviour, not weights. It imitates the teacher’s outputs on the inputs it sees and rarely matches it everywhere.
• Access is what differs. Logit and feature distillation need internal access; API distillation uses only outputs, including reasoning where exposed.
• The US advisory is an allegation, not a ruling. NSA, FBI and CISA named six Chinese companies on 8 September 2026, and China rejected the claims.
• Anthropic’s 90 to 99% cut is platform-specific. It describes data harvested from Claude, measured by Anthropic, not global distillation.
• Terms of service draw the commercial line. Google and OpenAI both bar using outputs to build competing models, but breaching terms isn’t automatically theft.
• The economics still favour distillation. Cheaper inference, lower latency and on-device deployment explain why it’s everywhere.
• Buyers should ask about provenance. What teachers and data a model learned from now affects licensing and supplier risk.

Quick Navigation

Model distillation began as an engineering fix

The idea was formalised in a 2015 paper by Geoffrey Hinton, Oriol Vinyals and Jeff Dean, “Distilling the Knowledge in a Neural Network”. Ensembles of many models were more accurate than any single one, but too slow and expensive to deploy. They wanted to “compress the knowledge in an ensemble into a single model”.

Their answer was to train a small student on the probabilities a large teacher assigns to every possible answer, as well as on the correct label. Those probabilities, which the paper calls “soft targets”, carry information a plain label throws away. A teacher that says an image is 90% a 2, 9% a 7 and 1% a 3 is telling the student which mistakes are reasonable.

On MNIST, a small network trained normally made 146 test errors. The same network trained on the teacher’s soft targets made 74, close to the large model’s 67. On a commercial speech system, a single distilled model recovered most of the accuracy gain of a 10-model ensemble.

Distillation isn’t the same as the other ways of shrinking a model. Quantisation stores the same model’s weights in fewer bits. Pruning removes weights or whole parts of a network. Fine-tuning adjusts an existing model on new data. Distillation trains a different, usually smaller, model to imitate the behaviour of a larger one. The methods are often combined.

How model distillation actually works

Strip away the maths and model distillation has four steps.

1. Choose inputs. Collect prompts or examples that cover the tasks the student needs to handle.
2. Ask the teacher. Run each input through the teacher and record what it produces: probabilities, answers, sometimes step-by-step reasoning.
3. Train the student. Optimise the smaller model to match those outputs.
4. Evaluate. Test the student against the teacher and, more importantly, against the real task.

The paper’s key trick is temperature. A model’s raw scores, called logits, pass through a softmax function to become probabilities. Dividing the logits by a temperature above 1 flattens the distribution, so the small probabilities on wrong-but-plausible answers become visible. The student learns from that softer distribution, usually alongside the true labels.

A simple example: a teacher reading the review “Not bad, but not worth the price” might assign 60% negative, 35% neutral and 5% positive. A plain label just says “negative”. The soft version tells the student the review is mixed, which helps it handle similar cases later.

Two limits matter. The student never receives the teacher’s weights; it learns some of the teacher’s behaviour on the inputs it was shown. And it won’t reproduce everything. Results depend on the teacher, the inputs, the training objective, the student’s size and architecture, and what you evaluate it on. A small student can match a large teacher on a narrow task and fall short elsewhere.

Model distillation methods, from logits to API outputs

“Distillation” covers several techniques that need very different access to the teacher, which is exactly what the current dispute turns on.

Response-based distillation is the form most relevant to large language models today. A developer prompts a strong model, saves the answers, and fine-tunes a smaller model on them as synthetic training data. If the teacher shows its step-by-step reasoning, the student can also learn how each answer was reached.

API-based distillation never touches the teacher’s weights, but it can still capture a lot. Anthropic reported that a model distilled on full reasoning transcripts from Claude scored 80.4% on the SWE-bench Verified coding benchmark, against 71.8% when trained only on summarised transcripts. That’s a company-run test, but it shows why labs now guard their models’ reasoning more closely than their answers.

Why AI companies are fighting over model distillation

What the US advisory alleges

On 8 September 2026, the NSA, FBI and CISA published joint advisory AA26-251A, “China-Based Artificial Intelligence Companies Conducting Industrial-Scale Distillation Campaigns Against U.S. AI Companies”. It named DeepSeek, Moonshot AI, Alibaba, MiniMax, StepFun and Z.AI.

According to summaries by the Cloud Security Alliance and The Next Web, the agencies alleged campaigns against versions of Claude, GPT, Gemini and Grok from at least late 2024 to mid-2026, totalling billions of tokens across millions of exchanges. The methods described include fraudulent accounts with obscured metadata, bulk commercial subscriptions, jailbreak prompts designed to expose hidden reasoning, and “transfer stations”: grey-market API proxies that resell access and get around regional blocks. The agencies also said the campaigns ran “likely with Chinese government awareness”, and called distillation “the core, not merely a supplement” of these companies’ development.

The advisory isn’t a court finding. As reported, the activity it describes is querying deployed models at scale; none of the summaries we read mention access to model weights. The agencies recommended that AI providers detect anomalous usage, share threat intelligence across the industry, and consider subtly altering responses to suspected distillers.

China’s response

China rejected the allegations. In a statement to the Associated Press, reported by Al Jazeera, the Commerce Ministry accused the US of seeking a “monopoly of the AI industry”, said distillation is commonplace in the industry including among US companies, and warned of “resolute countermeasures”. Foreign Ministry spokesperson Mao Ning urged the US to “refrain from making unfounded accusations or smears”.

The named companies didn’t respond publicly to requests for comment in the days after the advisory, according to The Next Web and Al Jazeera. None has admitted the conduct described.

What Anthropic’s September report claims

Anthropic, which is a party to this dispute, made its own allegations two days later. Its September 2026 threat intelligence report described campaigns it attributed to seven China-based labs. As summarised in independent coverage, the largest was attributed to Alibaba: more than 151 million exchanges between May and July, which Anthropic called “the largest distillation attack we have ever measured”. It also attributed more than 23 million exchanges to Moonshot AI and about 12 million in 14 days to DeepSeek, and named Zhipu, Xiaomi, SenseTime and MiniMax.

Anthropic said labs reached Claude through networks of fake accounts paid for with false identities or stolen cards, and through proxy services that resell access and sometimes sell the saved exchanges on. It described a “cross-session replay” technique: saving a signed reasoning token from one response, then prompting Claude in a new session to rebuild reasoning that would otherwise have been summarised.

The most serious allegation concerns Moonshot. Anthropic said that in one 10-day period, Moonshot silently forwarded nearly 300,000 requests from its own Kimi customers to Claude through 5,380 fraudulent accounts, and served Claude’s answers as Kimi’s. If accurate, that’s a data-governance problem as well as an IP one. Kimi users’ prompts, some containing sensitive data according to Anthropic, went to a provider they hadn’t chosen. Anthropic says it doesn’t know whether those customers were told.

These are Anthropic’s findings from its own telemetry. They haven’t been independently verified, and the named companies hadn’t published detailed responses when we checked.

Anthropic’s October defences against model distillation

On 12 October, Anthropic published an update on its distillation defences. Its headline claim: “early evidence suggests that these methods have reduced the amount of data harvested by distillers by around 90 to 99%”. It also said the number of reasoning tokens extracted by distillers “is consistently 10 to 100 times lower than it was before”.

Read those numbers carefully. Anthropic estimates extracted tokens “on our own platform”, so they describe what it observes on Claude. They don’t show that global distillation fell by 90 to 99%, and they don’t cover other providers, data resellers or transcripts harvested before the defences existed. Anthropic doesn’t describe the estimation method in detail.

The safeguards fall into three groups:

• Controlling where reasoning goes. A feature called preserved thinking stops new API accounts, created from 31 August 2026, from editing earlier context. Anthropic also stopped earlier, less protected models from reading newer models’ reasoning, and with Claude Sonnet 5.5, limited reasoning transcripts to the organisation that made the request. That’s aimed squarely at cross-session replay.
• Detecting and blocking extraction. A two-stage classifier based on Anthropic’s constitutional classifier research screens for extraction attempts. Anthropic puts its false-positive rate at under 0.1%.
• Raising the cost of trying again. Accounts that repeatedly attempt extraction face stricter limits or bans. Since 24 September, Anthropic has also charged for requests its safeguards block.

Anthropic acknowledges trade-offs. Preserved thinking required some developers to update their integrations. A false-positive rate under 0.1% still means some legitimate requests get blocked, and since 24 September, billed. Anthropic says it has “turned away meaningful revenue and trillions of tokens’ worth of paying traffic”. It also concedes the goal of not affecting legitimate users hasn’t been fully met, and calls distillation “a cat-and-mouse game”. Researchers who study model reasoning may also find the limits get in their way.

Anti-distillation is becoming part of platform security, alongside fraud and abuse prevention. Reasoning traces in particular are now treated as assets to protect.

When model distillation becomes a question of authorisation

The five scenarios below use near-identical techniques. What separates them is access, permission and the terms that apply.

Scenario 1 is uncontroversial; labs routinely distil their own large models into smaller ones. Scenario 2 is the same with a contract.

Scenario 4 is where most real ambiguity sits. Major providers restrict it in their terms. Google’s service terms for AI/ML services say customers won’t use generated output “to develop a similar or competing product or service”, and separately prohibit using output to “create or improve models similar to a Google Model”. OpenAI’s terms of use prohibit using output “to develop models that compete with OpenAI”. Those are contract terms. Breaking them can mean suspended accounts or a breach-of-contract claim, but whether it amounts to theft is a separate question.

The legal picture is unsettled. Copyright protection for AI-generated outputs is uncertain in many jurisdictions. Trade-secret claims may depend on whether a provider took reasonable steps to keep material like reasoning traces secret, which is one reason labs now hide or summarise them. Rules also differ between the US, China and the EU. Nothing here is legal advice. Anyone planning to train on another provider’s outputs needs a lawyer to read the specific terms.

Scenario 5, as the advisory and Anthropic describe it, is different in kind. Fake identities, stolen payment cards, proxy resale and covert relaying of customers’ prompts aren’t distillation techniques. They’re ways around controls, and they raise fraud and privacy questions whatever anyone thinks about training on outputs.

What model distillation means for AI buyers

For a company choosing a model, the dispute turns into a provenance question. If a model was trained substantially on another provider’s outputs in breach of its terms, the buyer could face uncertainty about licensing, continued availability or sanctions in some markets. That’s an assessment for each buyer, not a conclusion about any named model.

Our comparison of Chinese open-weight models found that quality at the top has converged, so licensing, hardware and customer identity now drive the choice. Provenance belongs on that list. Ask suppliers what data and teacher models a model was trained on, whether third-party outputs were used and under what terms, and what indemnities they offer.

Distilled models also raise the same trust questions as any modified model. A student can inherit a teacher’s biases or safety gaps, or lose safeguards the teacher had. As our look at editing AI models through machine unlearning argued, a changed model needs its own behavioural and security testing. Don’t assume it inherits the original’s evaluation.

The economics of model distillation

The controversy doesn’t change why organisations distil. Smaller models are cheaper and faster, need less memory, run on phones or on-premises hardware, and cut dependence on expensive frontier APIs.

Here’s an illustrative example. Every number is an assumption, not a measured figure. A company answers 1 million routine support enquiries a month with a large teacher model, at an assumed $0.02 per enquiry: $20,000 a month. It decides to distil a small student:

On those assumptions, the student saves $17,000 a month and pays back its one-off cost within the first month. But the example only works if the student is good enough. If it handles 80% of enquiries well and the rest must go back to the teacher, monthly costs rise to about $7,000, still well below $20,000. If quality drops and customers notice, the savings may not be worth it.

The example uses the company’s own teacher, which is scenario 1. Generating the same 200,000 answers from a third-party API to build a competing product could breach that provider’s terms. The economics are identical. The risk isn’t.

Open weights change the maths again. When a teacher is released under a permissive licence, distilling it can be explicitly allowed. As our coverage of Inkling 975B’s open weights noted, permissive weights can support training a smaller student, subject to the licence.

What comes next for model distillation

Expect platform controls to keep tightening. Hiding or summarising reasoning, binding reasoning to the organisation that requested it, detecting extraction patterns and suspending suspicious accounts are now standard tools, and the advisory encourages providers to share intelligence about distillers. The awkward recommendation is subtly altering responses to suspected distillers, which risks degrading answers for legitimate users caught by mistake.

Three open questions will shape the field. Can providers protect reasoning without making their models less useful to researchers and developers? Will courts treat bulk API extraction as a contract dispute, a trade-secret case or something else? And will governments act on the advisory through sanctions or procurement bans, given China’s warning of countermeasures?

The technique itself isn’t in question. Model distillation is how much of the industry turns expensive models into practical ones. The fight is over access, permission and what a contract allows.

FAQ

What is model distillation in AI?

Model distillation is a training technique in which a smaller “student” model learns to imitate a larger “teacher” model’s outputs. It’s used to make models cheaper, faster and easier to deploy.

How does knowledge distillation work?

The teacher produces outputs, such as probability distributions or answers, for a set of inputs, and the student is trained to match them. Softened probabilities, called soft targets, carry more information than single correct labels.

What is the difference between model distillation and fine-tuning?

Fine-tuning adjusts an existing model on new data. Model distillation trains a separate, usually smaller, model to copy another model’s behaviour, often by fine-tuning it on the teacher’s outputs.

Is model distillation the same as model compression?

It’s one form of model compression. Quantisation and pruning shrink the same model, while distillation trains a new, smaller one.

Can a smaller AI model learn from a larger model?

Yes, on the tasks it’s trained for. A distilled student can come close to its teacher on narrow tasks but usually falls short on broad ones.

Is using another AI model’s API outputs for training legal?

It depends on the provider’s terms and the jurisdiction. Many providers, including Google and OpenAI, prohibit using outputs to build competing models, so get legal advice before trying it.

Why are AI companies concerned about distillation attacks?

Because rivals can capture expensive capabilities, especially reasoning, by querying a model at scale. Labs also say covert campaigns rely on fraud and can expose customers’ data.

What did Anthropic announce about distillation defences in October 2026?

On 12 October, Anthropic said new safeguards cut the data harvested by distillers on its platform by around 90 to 99%. The measures include limits on where reasoning can go, extraction classifiers and penalties for repeat attempts.

How does model distillation reduce AI inference costs?

A smaller student needs less compute and memory per request than its teacher. For high-volume, repetitive tasks, that can cut running costs substantially once the one-off training cost is paid.

What should businesses check before deploying a distilled model?

Check where the training data and teacher outputs came from, the licence and terms that applied, and the model’s own safety and quality testing. Ask the supplier about indemnities.

Keep reading

Here are the latest posts from the blog.

Anthropic IPO: The $895 Billion Question Behind Its Valuation

November 9, 2026

Last updated: 9 November 2026. Latest confirmed development: Anthropic’s amended S-1, filed on 2 November, still leaves the share count and the IPO price range …

Read more

Claude Opus vs Sonnet: What the 2026 Price Gap Actually Buys

November 8, 2026

Last verified: 8 November 2026. All prices are Anthropic’s standard first-party API rates in US dollars per million tokens. They aren’t Claude Pro or Max …

Read more

Did AI Break RSA? What DeepMind Actually Found

November 8, 2026

Last verified: 8 November 2026. Both papers are unreviewed working papers, and no outside group has reproduced the results yet. No, AI did not break …

Read more

Model Distillation Explained: When AI Learning Becomes Extraction

November 7, 2026

Last verified: 7 November 2026. Government and company claims are attributed throughout. None has been tested in court. A support team trains a small, cheap …

Read more