AI distillation: AI companies face new security challenges Links to this article Published by Sarah Chaney Thursday, 28 September 2023 US-China relations have entered a new phase. For more than five years, the dispute has focused mainly on restrictions on how advanced chips can be manufactured or sold. Now attention is turning to something less visible but perhaps even more important: a new front in the technology rivalry in the form of large-scale use of AI to train other AI models.
What is AI distillation?
Artificial intelligence, or AI, distillation is a technique used to transfer the strengths of a large, high-quality or high-capacity system into a different model. The large system is called the teacher, while the system that is learning from the outputs of that model is the student. Instead of creating each system independently, developers can expose the student to outputs or examples generated by the teacher model and have those samples used to train the student. This can be a cost-effective way of creating an AI model.
For example, a smaller model might learn to perform certain tasks without having the same resources or infrastructure as the larger system.
The approach is not new and, on its own, does not constitute evidence of wrongdoing. Anthropic has acknowledged thatdistillation is used legitimately in the industry.
What’s changed?
In the dispute, scale and access are key; not so much the technology itself. US authorities have accused six Chinese AI companies of using access to American models on a large scale to accelerate their development efforts – what the US has described as “large-scale” or even “malicious” distillation. Washington has alleged that this involved methods intended to undermine controls on access to American systems.
China, for its part, has rejected the accusations and argued thatdistillation is used widely in the industry and is nothing sinister.
That distinction matters because legitimate distillation and unjustified capability extraction can be very different.
Why does AI distillation matter?
For one thing, it’s cheaper to develop a frontier AI model this way. To develop a comparable system independently would require vast amounts of computing resources, expertise and energy.
Using distillation, a company can use the outputs of an existing system as training data and spend its resources on training another system to reproduce specific skills.
That system would not necessarily be an exact replica of the original, but it could perform better on similar skills like data analysis or coding. For a second, it’s attracted a lot of attention.
How do the allegations unfold?
Anthropic has provided some of the most detailed recent allegations. The company said in a threat intelligence report in September 2026 that it had detected and disrupted additional distillation efforts involving seven China-based laboratories – “attempts to extract functionality from Anthropic’s Claude models at significant scale”. The company claimed some operations used hundreds or thousands of accounts and proxies to access the company’s models, presumably so that their outputs could then be used as training material in other AI models.
Anthropic claimed the activity involved millions of exchanges with its models.
It is worth noting that the scale at which the company says the activity occurred is based on its own investigation and the company has not disclosed information on the scale at which other providers have experienced attacks.
Where is this heading?
The dispute is becoming a part of the US-China AI rivalry in a broader sense, with advanced AI increasingly seen as a strategic asset. The US already restricts China’s access to some semiconductor technology that is used in developing advanced AI models, and the use of AI models as an intermediary allows that technology to be acquired indirectly.
US officials have suggested that these large-scale models can give Chinese developers an advantage in the race by reducing their costs and timelines, Reuters reported.
The US has also expressed security concerns about the use of Chinese cyber operations to train AI models; in particular, Reuters reported that such operations could be used for military and cyber-attack purposes. China has dismissed the allegations, but the allegations are part of a broader “conflict” between the two powers rather than independently verified facts. 1 Artificial intelligence With this in mind, AI companies face a new challenge in the form of identifying and defending against focused efforts to extract model capabilities.
An AI model is typically accessible via an application or application programming interface (API), and if an attacker exchanges very large volumes of questions it may be possible to extract responses that become useful training data. Companies will need to distinguish between genuine users and bad actors. According to Anthropic, it has responded by detecting and blocking suspicious accounts, detecting proxy networks and developing tools to identify adversarial extractions.
It has also added precautions to make the outputs of stolen models less useful for training new models.
What’s next?
The competition will influence how companies set access controls and plug-in fee models to optimize profits. And it could lead to China re-evaluating the concepts of intellectual property and intellectual property protection, especially as the focus on hardware restrictions that ban sale of high-performance chips is shifting to outputs of AI models themselves. The US and China are due to debate AI safety and progress this month, and Reuters reported that issues such as model theft were likely to be a part of that conversation.
For the AI industry, the broader lesson is clear: this is a phase of the AI race where success is increasingly defined not by who can train the biggest model but by who can develop powerful models efficiently, defend intellectual property and use it at scale.
Artificial intelligence Friday, 29 September 2023 Author Digital security, Technology security For the AI industry, the broader lesson is clear: this is a phase of the AI race where success is increasingly defined not by who can train the biggest model but by who can develop powerful models efficiently, defend intellectual property and use it at scale. As the world’s focus on AI grows more intense, it is possible that issues related to model distillation will continue to be one of the main fault lines in the evolving US-China technology relationship.
