vLLM Says It Matches DeepSeek’s Own Engine on V4.1-Flash Decode
Within 6% of DeepInfer from batch 8 to 256, its blog says; one GB300 NVL72 rack reaches 954K tokens/s per GPU.
@vllm_project says vLLM matches DeepInfer, DeepSeek’s own engine for DeepSeek-V4.1-Flash, on DeepSeek’s decode configuration across batch sizes 8 to 256. It does so at DEP32 against DeepInfer’s EP64DP64.
vLLM’s blog gives the detail. At 131K context, its decode throughput on GB200 is within 6% of DeepInfer’s published results. The DeepInfer figures were read off charts in its README.
@NeoAIForecast traces the loop. DeepInfer was originally built on vLLM. DeepSeek open-sourced its kernels, and vLLM pulled them back in. When the code is open, he writes, speed leads get shorter.
| Measure | Result |
|---|---|
| Decode gap to DeepInfer, batch 8 to 256 | Within 6% |
| One GB300 NVL72 rack, per GPU | 954K tokens/s at P90 196 tokens/s per user |
| Gain over its single-node result | 7.9× |
| Specialized decode path, one user | 826 tokens/s |
The blog cautions that the prefill setups differ, so it does not read the prefill gap as an engine-to-engine speedup. @MiaAI_lab reposted the news.
Sources
- @vllm_project on X · Oct 24, 10:59 AM PT
- @NeoAIForecast on X · Oct 24, 1:54 PM PT
- @MiaAI_lab on X · Oct 24, 3:20 PM PT
- vLLM blog · vllm.ai