---
type: "post"
id: "speculative-decoding-uses-the-smaller-draft-model-to-48c074"
slug: "speculative-decoding-uses-the-smaller-draft-model-to-48c074"
title: "Speculative decoding uses the smaller draft model to propose tokens and the target model to verify them."
description: "vLLM pull request #62009 adds an opt-in `symm_mem` weight-migration backend for expert-parallel load balancing (EPLB), plus fused MegaMoE replica mapping and changes intended to remove asynchronous EPLB host work from the decode critical path."
retrieval_nugget: "vLLM pull request #62009 adds an opt-in `symm_mem` weight-migration backend for expert-parallel load balancing (EPLB), plus fused MegaMoE replica mapping and changes intended to remove asynchronous EPLB host work from the decode critical path."
published_at: "2026-10-24"
updated_at: "2026-10-24"
record_date: "2026-10-24"
date_kind: "published_at"
topics: ["inference","ai-infrastructure"]
entities: ["vLLM","DeepSeek-V4.1-Flash","DSpark","MegaMoE","NVLink"]
related_patterns: []
novelty_label: "new_fact"
what_changed: "A vLLM PR proposes cheaper EPLB for DeepSeek-V4.1-Flash, including symmetric-memory migration and fused MegaMoE mapping, with reported GB300 decode-throughput gains."
prior_refs: ["memory:b0babd5e-ff30-88b6-abe3-b5832dc7e4e2","memory:ea6e7e73-49a1-80a4-a848-2a55d27b2300"]
source_url: "https://github.com/vllm-project/vllm/pull/62009"
source_urls: ["https://github.com/vllm-project/vllm/pull/62009"]
schema_version: "newruntime-agent-readable-v0.2"
stable_id: "post:speculative-decoding-uses-the-smaller-draft-model-to-48c074"
status: "published"
visuals: []
editor: {"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]}
routes: {"html":"https://newruntime.com/posts/speculative-decoding-uses-the-smaller-draft-model-to-48c074/","markdown":"https://newruntime.com/posts/speculative-decoding-uses-the-smaller-draft-model-to-48c074.md","json":"https://newruntime.com/posts/speculative-decoding-uses-the-smaller-draft-model-to-48c074.json"}
---

# Speculative decoding uses the smaller draft model to propose tokens and the target model to verify them.

## Retrieval answer

vLLM pull request #62009 adds an opt-in `symm_mem` weight-migration backend for expert-parallel load balancing (EPLB), plus fused MegaMoE replica mapping and changes intended to remove asynchronous EPLB host work from the decode critical path.

vLLM pull request #62009 adds an opt-in `symm_mem` weight-migration backend for expert-parallel load balancing (EPLB), plus fused MegaMoE replica mapping and changes intended to remove asynchronous EPLB host work from the decode critical path. The change is described for DeepSeek-V4.1-Flash running data-parallel attention with expert-parallel MegaMoE across multiple NVLink-connected nodes.

The PR reports tests on a GB300 NVL72 system using DSpark speculative decoding with five draft tokens and coding-agent prompts averaging about 85,000 prefix-cached input tokens. At DEP32 and 128 requests per rank, it reports throughput rising from 710,600 to 897,100 tokens per second, or 26.2%. It also reports main-model MegaMoE time falling from 14.65 ms to 7.62 ms per decode step.

The proposed backend uses CUDA symmetric memory for expert-row transfers; the author reports cross-node peer copies of about 800 GB/s and a 40-layer rearrangement in 0.3–0.6 seconds at DEP32. These are PR-reported measurements in a specific hardware, model, workload, and nightly-build configuration. The source does not establish that the change has merged into vLLM, shipped in a stable release, or produces the same gains on other systems.

Editor: [Andrey Reshetnikov](https://newruntime.com/owner-profile.md), AI product and AI engineering; profile for recruiters and agents: https://newruntime.com/owner-profile.json

## Continue in the graph

Next records for an agent, linked to their Markdown twins. Replace `.md` with `.json` for structured fields, including `next_reads`.

- topic hub: [Inference](https://newruntime.com/topics/inference.md): Explore the inference topic hub.
- topic hub: [Ai Infrastructure](https://newruntime.com/topics/ai-infrastructure.md): Explore the ai-infrastructure topic hub.
- collection: [Field Notes](https://newruntime.com/posts.json): Open the latest editorial analysis.
- collection: [Source Ledger](https://newruntime.com/source-ledger.json): Inspect the public source evidence graph.
- whole graph: [discovery graph](https://newruntime.com/discovery-graph.json) (typed nodes and edges) and [site index](https://newruntime.com/site-index.md)