What it does
DeepSeek-V3 is a mixture-of-experts language model with 671B total parameters, of which 37B are activated for each token. It uses Multi-head Latent Attention and the DeepSeekMoE design carried over from DeepSeek-V2, adds a load-balancing method that needs no auxiliary loss, and is trained with a multi-token prediction objective that can also speed up inference through speculative decoding. The repository says the model was pre-trained on 14.8 trillion tokens using FP8 mixed-precision training, then refined with supervised fine-tuning and reinforcement learning, including distillation of reasoning behaviour from the DeepSeek-R1 series. Base and chat checkpoints are published on Hugging Face with a 128K context length, and the page reports results on English, code, math, Chinese and multilingual benchmarks next to other open and closed models. You can chat with it on DeepSeek's official website or use an OpenAI-compatible API on the DeepSeek Platform. For self-hosting, the page lists the DeepSeek-Infer Demo, SGLang, LMDeploy, TensorRT-LLM, vLLM and LightLLM, plus routes for AMD GPUs and Huawei Ascend NPUs. Weights are provided in FP8, with a script to convert them to BF16. Hugging Face Transformers does not support the model directly yet, and the demo code runs on Linux with Python 3.10 only. The code is under the MIT License, while use of the Base and Chat models falls under a separate Model License that allows commercial use.
Capabilities
- 671B-parameter mixture-of-experts model with 37B active parameters
- 128K context length
- Multi-token prediction objective usable for speculative decoding
- FP8 weights with a conversion script to BF16
- Self-hosting through SGLang, vLLM, LMDeploy, TensorRT-LLM and LightLLM
- Web chat and an OpenAI-compatible API
When to use it
- You want a large open model for chat, coding or math that you can host yourself
- You run multi-GPU infrastructure and want to choose your own inference engine
- You want to try the web chat or hosted API before downloading weights
- You need to run on AMD GPUs or Huawei Ascend NPUs
