vLLM and SGLang can match DeepInfer per kernel step, but their benchmarks don't show production parity

What we checked

• Do the vLLM, SGLang, and DeepInfer results use equivalent models, hardware, batch sizes, latency targets, and end-to-end serving configurations? partly confirmed. All three use DeepSeek-V4.1-Flash. SGLang's decode test is the closest match: it ran DeepInfer's own decode workload on the same 64 GB300 GPUs at EP64, with 8–256 requests per GPU. vLLM ran decode on GB200 against DeepInfer's 'SM100' charts, and its prefill runs use a different parallelism layout, different redundant-expert and replay settings, and the authors' own prompts. None of the three states a latency target, and none compares full serving stacks, because public DeepInfer leaves out the KV cache system, prefill/decode disaggregation and load balancing. [3][2][1]
• Is vLLM's $38.96/GPU-hour a like-for-like match for DeepSeek's reported $38/GPU-hour? refuted. vLLM's figure comes from SemiAnalysis's AgentX agentic traces on one GB300 NVL72 rack, priced at DeepSeek's off-peak rates. DeepSeek's $38 is the overall revenue of a production cluster serving 155 trillion tokens a day, also at off-peak pricing. The two workloads differ, as the source summary itself notes. [2][4][6]
• Is the open-source DeepInfer the same as DeepSeek's production engine? partly confirmed. DeepSeek says the public kernels are 'broadly the same', but public FlashMLA lacks one hand-tuned SASS optimization, which makes decode 5–10% slower. It puts the end-to-end gap at 'at most 5%'. The KV cache and load-balancing code is not fully open-sourced, so 'matching DeepInfer' is not the same as matching DeepSeek production. [4][1]

What we found

• DeepInfer's own benchmark section says only that results were 'measured on 4 SM100 GPUs for prefill and 64 for decode — each within one scale-up domain.' The charts are images with no numbers, and the README gives no parallelism, batch-size, context or speculative-decoding settings for them; it points to a run_sweep.sh script instead. Both competitors read DeepInfer's values off those charts by eye rather than from raw data. [1][2][3]
• SGLang's comparison is the most controlled. It ran DeepInfer itself (commit 774ce24) against SGLang on the same hardware and weights: 4x GB300 and 4x B300 for prefill at DEP4, and 64x GB300 at EP64 for decode. Decode used 128K context, 3 draft tokens per step with static verification, and 8–256 requests per GPU. The setup still differs in places. Balanced routing was used 'for the comparison only'. The engines place Engram tables differently (SGLang on device, DeepInfer in host memory). DeepInfer's draft model uses 64 redundant experts and SGLang's uses none. The prefill timing windows and prompt construction also differ. SGLang published its commits and launch commands. [3]
• vLLM's decode claim, within 6% at batch sizes 8–256, uses DeepSeek's 131K-context, nextn=3 setup, but runs on GB200 against charts that say only 'SM100'. For prefill, vLLM itself warns that DeepInfer used EP4DP4 with 32 redundant experts and a 140-token replay, while vLLM used DEP2 on GB300 with no redundant experts, a 128-token replay and its own prompts. It says the gap should not be read as an engine-to-engine speedup. [2]
• vLLM's headline number is a full-system result on a different workload: 954K tokens/s per GPU at P90 196 tokens/s per user, on one GB300 NVL72 rack with prefill/decode disaggregation and Wide-EP up to DEP32 with EPLB, running SemiAnalysis's AgentX agentic traces. Throughput counts cached input, uncached input and output tokens. The post reports P90 per-user speed but no explicit SLO, and it does not give exact input/output lengths for the AgentX runs. [2]
• The public DeepInfer cannot run a full production service by itself. The README lists a KV cache system, VE/Prefill/Decode disaggregation and DP load balancing as 'Required for a complete inference service (not included in this release).' DeepSeek's Q&A adds that public FlashMLA decode is 5–10% slower than the internal kernel and that end-to-end performance is 'at most 5% apart'. Production uses EP64–128 for decode on supernodes and at least 64 prefill plus 64 decode GPUs. [1][4]
• Mainstream coverage such as The Decoder describes the release and notes that DeepInfer builds on vLLM. It reports no independent benchmarks. I found no third party that reproduced either competitor's claims. [5]

What it means

The defensible reading is narrow. For V4.1-Flash on Blackwell, SGLang has shown per-step decode and prefill throughput at or slightly above the open-source DeepInfer. It did so on matched hardware and on DeepInfer's own benchmark workload, using settings chosen for the comparison and not SGLang's defaults. vLLM has shown decode in the same range on GB200, plus a strong full-rack AgentX result. Neither result shows that a general-purpose engine matches DeepSeek's production service. DeepInfer's public charts lack exact settings, the production cache and load-balancing layers are not public, and nobody states a latency target. The $38 versus $38.96 comparison should be treated as coincidence across different workloads, not as cost parity.

For builders, the engine choice probably depends less on raw kernel speed than on everything around it. DeepSeek's kernels and DSpark speculative decoding are already merged into SGLang's main branch, and vLLM ships disaggregation, EPLB and broad model support, all things DeepInfer lacks. If you must decide, run the published scripts yourself (DeepInfer's run_sweep.sh, SGLang's gist, vLLM's bench tooling) on your own traffic mix, cache-hit rate and per-user speed target, and compare complete systems at equal latency.

Still open

• What exact parallelism, batch, context and draft settings produced DeepInfer's README charts, and on which SM100 part (B200, B300, GB200 or GB300)?
• How do the engines compare as complete systems (cache, disaggregation, load balancing) at a stated latency target on the same real traffic mix?
• How much of SGLang's edge survives with its default settings, unbalanced real routing and identical Engram placement?
• Would DeepSeek's internal engine, with its hand-tuned FlashMLA, erase the 1–7% decode gaps SGLang reports?