---
type: "post"
id: "deepseek-v4-1-flash-on-vera-rubin-nvl72-0d61a3"
slug: "deepseek-v4-1-flash-on-vera-rubin-nvl72-0d61a3"
title: "DeepSeek-V4.1-Flash on Vera Rubin NVL72: 58x Throughput vs. GB300 NVL72 at 387 tokens/s"
description: "Inferact and the vLLM team report serving DeepSeek-V4.1-Flash on one NVIDIA Vera Rubin NVL72 rack at 1.7 million tokens per second per GPU, with 387 tokens per second per user in the SemiAnalysis AgentX benchmark."
retrieval_nugget: "Inferact and the vLLM team report serving DeepSeek-V4.1-Flash on one NVIDIA Vera Rubin NVL72 rack at 1.7 million tokens per second per GPU, with 387 tokens per second per user in the SemiAnalysis AgentX benchmark."
published_at: "2026-11-13"
updated_at: "2026-11-13"
record_date: "2026-11-13"
date_kind: "published_at"
topics: ["inference","compute","models"]
entities: ["vLLM","Inferact","DeepSeek-V4.1-Flash","NVIDIA Vera Rubin NVL72","NVIDIA","Dynamo","Mooncake","SemiAnalysis AgentX"]
related_patterns: ["agent-economics-moves-to-completed-work"]
novelty_label: "new_fact"
what_changed: "vLLM and Inferact report porting DeepSeek-V4.1-Flash to Vera Rubin NVL72, with 387 tokens/s per user and 58× the throughput of GB300 NVL72."
prior_refs: ["https://newruntime.com/posts/vllm-blog-serving-deepseek-v4-1-flash-at-073f14/","https://newruntime.com/posts/vllm-adds-day-0-support-for-nvidia-vera-aaa9ff/"]
source_url: "https://vllm.ai/blog/2026-11-12-deepseek-v41-flash-vera-rubin"
source_urls: ["https://vllm.ai/blog/2026-11-12-deepseek-v41-flash-vera-rubin"]
schema_version: "newruntime-agent-readable-v0.2"
stable_id: "post:deepseek-v4-1-flash-on-vera-rubin-nvl72-0d61a3"
status: "published"
visuals: []
editor: {"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]}
routes: {"html":"https://newruntime.com/posts/deepseek-v4-1-flash-on-vera-rubin-nvl72-0d61a3/","markdown":"https://newruntime.com/posts/deepseek-v4-1-flash-on-vera-rubin-nvl72-0d61a3.md","json":"https://newruntime.com/posts/deepseek-v4-1-flash-on-vera-rubin-nvl72-0d61a3.json"}
markdown_url: "https://newruntime.com/posts/deepseek-v4-1-flash-on-vera-rubin-nvl72-0d61a3.md"
---

# DeepSeek-V4.1-Flash on Vera Rubin NVL72: 58x Throughput vs. GB300 NVL72 at 387 tokens/s

## Retrieval answer

Inferact and the vLLM team report serving DeepSeek-V4.1-Flash on one NVIDIA Vera Rubin NVL72 rack at 1.7 million tokens per second per GPU, with 387 tokens per second per user in the SemiAnalysis AgentX benchmark.

## Summary
Inferact and the vLLM team report serving DeepSeek-V4.1-Flash on one NVIDIA Vera Rubin NVL72 rack at 1.7 million tokens per second per GPU, with 387 tokens per second per user in the SemiAnalysis AgentX benchmark. The result matters because the report ties its gains to rack-scale memory pooling and workload routing as well as GPU kernels, while showing that serving performance depends on the whole system.

## What happened
vLLM’s November 12, 2026 announcement describes a port of DeepSeek-V4.1-Flash to a Vera Rubin NVL72 rack. The team says it measured performance using SemiAnalysis AgentX, a representative agentic serving workload, with the same methodology used in its earlier GB300 NVL72 study. Its headline result is 1.7 million total tokens per second per GPU while each user receives 387 tokens per second; peak throughput reaches 2.52 million tokens per second, and a specialized single-user path reaches 1,810 tokens per second at P90. The report’s throughput includes cached input, uncached input and output tokens.

The engineers say Rubin’s hardware shaped the port. Each GPU has two memory domains, with faster access to the domain local to an SM; the SM107 architecture adds wider block-scaled matrix instructions; and all 72 GPUs share one NVLink domain. The team reworked MoE, attention, GEMM, mHC and indexer kernels around those properties. It reports that DEP2 prefill steps are about 28% faster and DEP decode steps about 25% faster. In a low-latency path, tensor parallelism across two compute trays reached 1,810 tokens per second at P90 for one user. These are workload- and configuration-specific measurements, not a general guarantee for other models or serving stacks.

The report also describes moving beyond per-rank caches. A Mooncake KV pool spans DRAM and spare HBM, with data loaded over RDMA; Engram tables are sharded across GPU HBM and read over NVLink. The HBM extension of the KV pool is explicitly described as experimental. With pooled prefixes, sessions no longer need to remain pinned to one rank, and Dynamo routes each request to the worker and data-parallel rank with the least uncached work. On three DEP2 instances at concurrency 4,800, vLLM reports 68% more requests served, median time to first token falling from 29.8 seconds to 4.2 seconds, and cache hit rate rising from 91.8% to 96.1%.

The engineering details suggest why the headline is not a simple hardware comparison. For example, the team says asynchronous pool lookups and earlier KV loads reduced prefill time lost to underfilled steps, while workload-aware routing reduced rank imbalance. It also reports a 12-by-DEP2 simulation in which idle time caused by one rank waiting while another queued fell from 10.3% to 1.3%, and P90 time to first token fell from 0.98 seconds to 0.38 seconds. The announcement credits DeepSeek for open-sourcing the model and kernels, NVIDIA for early access to Vera Rubin NVL72, and SemiAnalysis for the AgentX benchmark.

## What the archive adds
New Runtime’s earlier coverage of vLLM’s day-zero Rubin support noted that runtime readiness can arrive before production evidence ([October 10 note](https://newruntime.com/posts/vllm-adds-day-0-support-for-nvidia-vera-aaa9ff/)). An October 27 note covered the GB300 NVL72 serving study ([earlier performance note](https://newruntime.com/posts/vllm-blog-serving-deepseek-v4-1-flash-at-073f14/)). This is a new performance result on Vera Rubin, extending a known serving story rather than merely restating it.

## Trend signal
This supports [agent economics moves to completed work](https://newruntime.com/patterns/agent-economics-moves-to-completed-work/): the report evaluates throughput and latency on an agentic workload, not just model calls. It also shows why the relevant unit is not yet cost per completed task: the announcement does not report task success, cost, or quality alongside the throughput figures.

## Why now
The report follows vLLM’s October GB300 study and arrives after its earlier account of day-zero Rubin support. It provides a first detailed description of how the same serving stack changes for the Vera Rubin rack, including memory pooling and routing at higher concurrency.

## Connections to other events
New Runtime’s October 24 coverage described DeepSeek-V4.1-Flash serving on GB300 NVL72, including prefill/decode disaggregation and Wide-EP. The new report keeps those engine shapes and moves them to Vera Rubin, while adding Rubin-specific kernels and rack-wide pools. It therefore connects model-serving benchmarks to the operational problem of sharing cache capacity across a rack.

## Why it matters
For teams planning large-scale inference, the report is a reminder that accelerator peak performance is only one part of serving economics. At high concurrency, cache placement, rank balance and memory traffic can determine whether faster GPUs translate into lower latency and more requests served. vLLM’s reported gains come from a coordinated stack: optimized kernels, pooled KV and Engram data, and routing that avoids session affinity. That offers a useful checklist for evaluating a serving deployment, but not a portable performance forecast. The benchmark uses one model, one workload and a specific Rubin configuration; teams need measurements on their own request mix, cache behavior, hardware and latency targets before making capacity or cost decisions.

## Limits and uncertainty
These are results reported by the vLLM team, which led the implementation with Inferact; they are not an independent replication. The headline figures use SemiAnalysis AgentX and a particular DeepSeek-V4.1-Flash configuration, so they do not establish comparable performance for other models, workloads or deployments. The announcement does not provide cost per successful task or model-quality comparisons alongside the throughput claims. It also labels the spare-HBM extension of the KV pool experimental, leaving its operational maturity and behavior outside this test unestablished.

## Skill to build now
If you operate a multi-GPU inference service, benchmark the system as a whole: record throughput, per-user decode speed, time to first token, cache-hit rate and rank imbalance at realistic concurrency. Then test whether pooling prefixes or changing routing improves your own workload before assuming a faster accelerator alone will deliver the gain.

## Reader takeaway
The reported 58-fold comparison is a specific AgentX result, not a universal Rubin multiplier. Its more transferable lesson is that serving gains came from coordinating kernels, memory domains, shared cache capacity and routing; teams should test those layers together against their own workloads.

Editor: [Andrey Reshetnikov](https://newruntime.com/owner-profile.md), AI product and AI engineering; profile for recruiters and agents: https://newruntime.com/owner-profile.json

## Continue in the graph

Next records for an agent, linked to their Markdown twins. Replace `.md` with `.json` for structured fields, including `next_reads`.

- topic hub: [Inference](https://newruntime.com/topics/inference.md): Explore the inference topic hub.
- topic hub: [Compute](https://newruntime.com/topics/compute.md): Explore the compute topic hub.
- topic hub: [Models](https://newruntime.com/topics/models.md): Explore the models topic hub.
- collection: [Field Notes](https://newruntime.com/posts.json): Open the latest editorial analysis.
- collection: [Sources](https://newruntime.com/sources/): See the original sources this archive cites.
- whole graph: [discovery graph](https://newruntime.com/discovery-graph.json) (typed nodes and edges) and [site index](https://newruntime.com/site-index.md)