vLLM Adds Day-Zero Kimi K3 Serving With Inferact’s DSpark Draft Model

vLLM announced day-zero serving support for Moonshot AI’s Kimi K3, with Inferact’s open-source DSpark draft model included as part of the launch.

Kimi K3 reaches vLLM on day zero

In a July 27 blog post, the vLLM project said it had added efficient day-zero support for Kimi K3, Moonshot AI’s large mixture-of-experts model. The accompanying vLLM Recipes page describes Kimi K3 as a 2.8-trillion-parameter MoE model with a 1-million-token context window, and provides Docker, hardware, and serving prerequisites for running the model through vLLM.

The release is focused on inference serving rather than model training. vLLM says its Kimi K3 support includes speculative decoding through Inferact’s DSpark speculator, a draft model intended to accelerate generation by proposing tokens that the target model can verify.

Inferact publishes Kimi-K3-DSpark on Hugging Face

Inferact’s Hugging Face model page describes Kimi-K3-DSpark as an “MLA-native DSpark draft model” for Kimi K3 on vLLM. The model card says the speculator was trained using TorchSpec and is designed for use with vLLM’s Kimi K3 serving path.

According to Inferact’s published model documentation, Kimi-K3-DSpark reached a peak batch-size-1 decode speed of 464 tokens per second on 4×GB300 hardware with tensor-parallel-size 16. That figure is presented by Inferact as a separate benchmark from vLLM’s own SPEED Bench numbers, so the two results should not be treated as directly interchangeable.

vLLM’s blog reports that DSpark improved its Kimi K3 SPEED Bench throughput from 118 tokens per second to 370 tokens per second. Magica’s coverage of the vLLM launch corroborates the same 118-to-370 tokens-per-second result and notes Inferact’s draft-model documentation around acceptance behavior.

Why the draft model matters

Speculative decoding can improve serving performance by letting a smaller draft model propose several candidate tokens before the larger target model verifies them. In this case, vLLM attributes its reported speedup to the DSpark draft model used with Kimi K3.

The sources do not establish that these speeds will be seen in all deployments. vLLM’s recipe page lists specific serving prerequisites, and Inferact’s model card ties its 464 tokens-per-second peak result to a particular hardware and parallelism setup. Performance will depend on serving configuration, batch size, hardware, and workload.

Still, the release is notable for making Kimi K3 available through a widely used open-source inference server immediately at launch, with an accompanying open draft model for speculative decoding. For teams evaluating Kimi K3, the vLLM blog, vLLM Recipes page, and Inferact’s Hugging Face model card now provide the main implementation details and benchmark references needed to reproduce or compare serving results.