Before celebrating how open and accessible modern AI has become, it’s worth asking a harder question: what happens when the very technique that makes AI affordable and portable is also the technique a foreign military uses to extract capabilities that were never meant to leave an American lab?
That question is no longer hypothetical. A Reuters review of more than 80 Chinese academic papers and patents found that researchers linked to the People’s Liberation Army (PLA) — China’s military — have been using outputs from OpenAI and Anthropic models to train specialized domestic AI systems for surveillance, cyber warfare, and drone targeting. The method they used is called model distillation, and understanding it is essential to understanding why this story matters.
The Concept Behind It
What Model Distillation Actually Is
Model distillation — sometimes called knowledge distillation — is a technique where a large, powerful AI model (the “teacher”) is used to generate outputs that then train a smaller, more efficient model (the “student”). The student doesn’t need to be as big or computing-hungry as the teacher; it just learns to replicate the teacher’s behaviour on a specific task.
Think of it like this: imagine a world-class chef writes out step-by-step instructions for a single dish, and a culinary student follows those instructions until they can cook that dish perfectly — without ever needing to understand every technique the chef spent decades mastering. The student can’t replicate the chef’s full repertoire, but on that one dish, they’re nearly indistinguishable.
In AI terms, the “dish” might be classifying social media content, recognizing objects in drone footage, or summarizing military source code. The student model becomes a lean, deployable tool — runnable on a laptop, a drone, or a device cut off from the internet — rather than a massive system requiring vast data centres.
Why the Industry Uses It
Distillation is genuinely useful and widely practiced across the AI industry. It’s one of the main reasons AI has moved from the cloud onto smartphones, medical devices, and industrial sensors. Smaller models are cheaper to run, faster to respond, and don’t require a live internet connection. Governments and companies around the world — including in the United States — use distillation routinely. China’s central government has actively promoted what it calls “model lightweighting” and edge computing, directing subsidies toward exactly this kind of efficiency research.
How the Pieces Fit Together
The Optimistic Framing
On the surface, the story of China using distillation sounds almost unremarkable. Of course researchers everywhere want to build efficient, locally deployable models. Of course they use available AI tools to generate training data. That’s just good engineering. And China’s AI companies — including firms like Moonshot and Alibaba, which have publicly denied relying on foreign model distillation — argue that their advances are driven by genuine domestic innovation.
The Overlooked Risks
The concern isn’t distillation as a concept. The concern is what was distilled, from what, and for whom — without authorization.
Researchers at PLA Unit 96941, a military intelligence and cyber-warfare unit in Beijing, published a paper describing how they used OpenAI’s GPT-3.5 to summarize sensitive military source code, then trained a domestic model on those summaries. The explicitly stated reason: third-party models were unsuitable for classified information, so they used a Western model as a processing intermediary and then cut it out of the loop entirely. That is not a routine engineering exercise. It is a deliberate method for extracting proprietary reasoning capabilities from a system whose developer has no idea how its outputs are being used.
At the North University of China — an institution with documented ties to China’s weapons industry — researchers used Anthropic’s Claude 3 Haiku to generate synthetic training data for a social media monitoring and content moderation system. Anthropic has stated it does not provide commercial access to Claude in China or to Beijing-controlled entities, and that it uses monitoring systems to detect policy violations. But the papers suggest those controls have not been fully effective. Crucially, Anthropic also warned that distilled models may lose the safety guardrails built into the original system — meaning capabilities can transfer while the limits designed to constrain them do not.
A 2024 paper from the PLA’s National University of Defense Technology described shrinking an image-processing model for deployment on unmanned aerial vehicles (drones), enabling real-time video analysis for navigation and targeting even when communications are severed. A study from the Academy of Military Sciences described running a target-recognition model on tactical hardware during simulated operations involving drones, ships, and unmanned submarines. The question of how AI makes targeting decisions without human oversight is already one of the most contested issues in international security — and this research pushes that frontier further.
What makes this pattern particularly significant is the convergence of two separate pressures: Washington’s export controls on advanced chips are designed to slow China’s ability to build frontier AI from scratch, but distillation offers a partial end-run around that constraint. By extracting the reasoning steps of frontier Western models into smaller, locally deployable systems, Chinese military researchers can potentially leapfrog the hardware bottleneck — not by matching American compute power, but by borrowing the intelligence it produced. This means chip export controls and AI model access controls need to be treated as a unified policy problem, not two separate ones.
Historical Parallels
This is not the first time a dual-use technology — one with legitimate civilian applications and serious military implications — has created a policy gap. Encryption, GPS, and semiconductor lithography all went through periods where their proliferation outran the regulatory frameworks designed to govern them. In each case, the technology moved faster than the rules. AI model distillation looks similar: a technique born in academic research, embraced by industry, and now available to anyone who can call an API — including people the API providers would not knowingly serve.
The dispute has already become a flashpoint ahead of U.S.-China talks on AI governance and safety. U.S. officials have accused some Chinese entities of using distillation to extract capabilities from American AI models, potentially undermining export controls and infringing intellectual property rights. China has rejected the accusations, arguing that Washington is pursuing AI “hegemonism” and that U.S. firms engage in similar practices. This geopolitical dimension is unlikely to resolve quickly.
What People Get Wrong
Misconception 1: Distillation Is the Same as Copying a Model
Distillation does not copy the underlying model — its weights, architecture, or parameters. It copies the behaviour of the model on specific tasks, by training a new model to produce similar outputs. This is an important legal and technical distinction: the student model is genuinely different from the teacher. But “technically different” does not mean “ethically or legally unproblematic,” especially when the outputs used for training were generated in violation of a platform’s terms of service or without the developer’s knowledge.
Misconception 2: Distilled Models Are Just as Powerful
They are not. As Trevor Koverko, co-founder of AI data company Sapien, put it, distillation is best understood as “transferring selected capabilities into a cheaper, locally controlled system, not achieving independence from frontier AI.” A distilled drone-targeting model is genuinely useful for a narrow military task. But it cannot replicate the broad intelligence of the frontier model it learned from. The student knows one dish, not the whole menu.
Misconception 3: This Is Only a Problem for AI Companies
It isn’t. If distillation can transfer reasoning capabilities from safety-tested Western models into unmonitored systems — stripping out the safety constraints in the process — then the failure of AI containment becomes a public concern, not just a corporate one. The same dynamic applies to AI systems approaching sensitive capability thresholds: what happens when those capabilities can be distilled into smaller, harder-to-monitor systems?
The Strongest Counterargument
The most serious pushback to this entire framing comes from researchers and China-watchers who argue that the threat is overstated — and that the papers themselves are being misread. Their point: academic papers describing the use of a Western model do not prove those models were accessed illegally or that the resulting systems are militarily significant. Chinese researchers routinely cite foreign tools in papers to signal methodological sophistication, and peer reviewers in the West have noted that claims in Chinese defence-linked academic literature sometimes exceed actual capability. Jamestown Foundation fellow Sunny Cheung, whose analysis underpins much of the Reuters investigation, works for an institution with an explicitly hawkish stance on China — a legitimate source, but not a neutral one.
This counterargument has genuine force. It suggests the story’s framing may conflate “researchers used GPT-3.5 as a tool” with “China’s military has now weaponized OpenAI’s reasoning.” Those are very different claims. What it does not weaken is the core finding: that there is a systematic, documented pattern of military-linked institutions using distillation as a deliberate strategy to build locally controlled AI from Western model outputs — regardless of how capable the results are. The policy gap is real even if the military threat is currently limited.
Where to Learn More
- Distilling the Knowledge in a Neural Network — the original 2015 paper by Geoffrey Hinton, Oriol Vinyals, and Jeff Dean that introduced the concept of knowledge distillation to the modern AI field.
- Anthropic’s Usage Policy — explains what Anthropic prohibits, including access from sanctioned regions and use for weapons development.
- U.S. Bureau of Industry and Security — CHIPS Act and Export Controls — the regulatory framework governing advanced semiconductor exports that forms the backdrop to this story.
The Open Questions
- Can AI model providers reliably detect when their systems are being used to generate distillation training data at scale, given that individual API calls look indistinguishable from legitimate use?
- If distilled models lose the safety guardrails of the original system, what obligation do frontier AI developers have to track and respond to that downstream risk?
- Should model distillation from a foreign military-linked institution without authorization be treated as a violation of export controls, intellectual property law, or both — and which body has jurisdiction?
- As China’s domestic frontier models close the capability gap with Western ones, will distillation from U.S. models become less strategically valuable — or will the practice simply shift to newer, more capable targets?
- What role should international AI governance talks play in defining acceptable uses of distillation across national borders, and is any such agreement verifiable?
