DeepSeek AI quickly conquered the mobile world, and now it’s expanding to Windows – with full support from Microsoft, which is surprising. The software giant added the DeepSeek R1 model to its Azure AI Foundry to allow developers to test and build cloud apps and services with it. Now, Microsoft has announced that it is adding distilled versions of R1 to Copilot+ PCs.
A distilled model in AI is a simplified version of a larger neural network that partially retains its performance but works faster and more efficiently. The distillation process allows transferring knowledge from a large, powerful model to a smaller one. In this case, the full DeepSeek R1 has 671 billion parameters, and smaller models will have between 1.5 billion and 14 billion parameters. It’s not a perfect process, and the distilled model is less powerful than the full version, but its smaller size allows it to run directly on consumer hardware (instead of specialized AI hardware that costs tens of thousands of dollars).
The distilled DeepSeek R1 models will be available first for devices based on Snapdragon X chips and Intel Core Ultra 200V processors, and then for PCs based on AMD Ryzen AI 9.
The first model will be the DeepSeek-R1-Distill-Qwen-1.5B (i.e., the 1.5 billion parameter model), with the larger and more powerful 7B and 14B models coming soon. They will be available for download from the Microsoft AI Toolkit.
Microsoft had to tweak these models to optimize them for running on NPU devices. Operations that are heavily dependent on memory access are performed on the CPU, while computationally intensive operations, such as the transformer block, are performed on the NPU. Thanks to Microsoft’s optimization, it was possible to achieve a fast time to first token (130 ms) and a throughput of 16 tokens per second for short hints (less than 64 tokens). Note that «token» is similar to a vowel letter (importantly, one token is usually more than one character long).
Microsoft is a strong advocate of AI and has invested in OpenAI (creator of ChatGPT and GPT-4o), but it seems to be looking to diversify. Its Azure Playground includes GPT (OpenAI), Llama (Meta), Mistral (an AI company), and now DeepSeek.
Source: gsmarena