Challenges in Stopping China's DeepSeek from Utilizing American AI Technology

This week, leading White House advisers expressed concerns that China’s AI company, DeepSeek, may have used a controversial method called "distillation" to enhance its technology. Distillation allows one AI system to learn from another, which raises alarms for officials who believe it may involve using advancements made by US competitors.

DeepSeek has recently made waves in the technology sector by introducing a new AI model that competes directly with products from major US companies like OpenAI. Notably, DeepSeek’s model is being offered at a significantly lower cost, with the company even providing its code for free.

There are concerns that DeepSeek’s model might have developed its capabilities by mimicking American AI systems. Experts familiar with the field suggest that DeepSeek could have utilized the distillation technique, where older, established AI models evaluate the performance of newer ones. This process allows the newer model to benefit from the substantial investment that went into creating the original, without incurring the same high costs.

Distillation is often seen as an essential practice in the AI domain, even though it might breach terms of service set by some American tech firms, including OpenAI. The makers of ChatGPT have reported that certain groups in China are actively trying to reproduce US AI models through distillation practices. They are currently assessing whether DeepSeek has operated outside the acceptable usage of their models.

Experts in the tech industry are skeptical about the US government's ability to prevent DeepSeek from leveraging this technique. Naveen Rao, the AI vice president at Databricks, pointed out that learning from competitors is a common practice in the industry. He drew parallels to how car manufacturers study and learn from each other's engines.

During a confirmation hearing, Howard Lutnick, who has been nominated to oversee export controls on AI technology, stated his belief that DeepSeek may have improperly used US technology. He assured the Senate that he would push for strict enforcement of regulations to protect US interest in AI.

Additionally, David Sacks, who serves as the White House’s AI and crypto advisor, echoed concerns about DeepSeek’s use of distillation in a recent interview, highlighting the serious implications for US technology.

While DeepSeek has not publicly responded to the accusations, OpenAI has committed to collaborating with the US government to safeguard American technology, but did not specify the measures they will take.

The recent worries about China benefiting from US technological advancements reflect previous tensions regarding the semiconductor industry, where the US has already placed restrictions on what can be exported to China.

The Complexity of Stopping Distillation

Experts suggest that stopping distillation might be more complicated than it seems. DeepSeek has demonstrated that even using a small number of data samples, less than one million, from a larger, more sophisticated model can significantly enhance a smaller model's abilities.

This makes it challenging to identify and regulate instances of distillation, especially when popular AI models, like ChatGPT, boast hundreds of millions of users. Consequently, monitoring such low data traffic becomes hard. Moreover, certain models developed by companies like Meta Platforms and the French startup Mistral are available for free and can be installed offline, complicating efforts to track potential violations.

Umesh Padval, managing director at Thomvest Ventures, stated that it is nearly impossible to halt model distillation using open-source models like Mistral and Llama. These models are easily accessible, and parties may discover US models through other customers.

Meta’s licensing for the Llama model mandates that users declare if they are employing it for distillation. DeepSeek has mentioned in a research paper that it used Llama to create some of the models it recently launched. However, they did not clarify if they utilized Meta’s model earlier in the development process.

There are calls from some AI laboratories for stringent regulations akin to know-your-customer rules in finance to limit the ability of firms like DeepSeek to engage in distillation.

However, such regulatory measures are yet to be implemented. Previous proposals from the Biden administration sought to establish similar regulations, but it's uncertain if the current leadership will adopt them.

Jonathan Ross, CEO of Groq, an AI computing company, has taken preventative measures by blocking all Chinese IP addresses from accessing his cloud services, aiming to prevent Chinese firms from utilizing the AI models hosted there. Nevertheless, Ross admitted that such actions might not be enough, as people can always find ways to bypass restrictions.

China, AI, DeepSeek