What happened
The EAGLE-4 release notes describe version 0.1.2 as an update to eagle-harness for the 0.1.1 engines. The central SGLang change is a per-batch decision about whether to speculate. A served tree, the default policy, estimates speculative value from tokens committed across served intervals and measures plain decoding; it then decodes a batch without speculation when that is faster. Requests handled this way stop drafting until they finish. Served chains use the same per-batch choice of verification width and also fall back to plain decoding when it wins.
The notes say the SGLang switch requires the radix cache to be disabled with --disable-radix-cache. With the cache enabled, the hybrid model tracks prefix-cache state that plain decoding does not carry, so the server speculates at every load and reports that in its log. The high-load recipe is for four GPUs; high load on one GPU is not supported. The ordinary SGLang quick-start becomes slower than plain decoding above roughly 32 to 64 concurrent requests on one GPU with tensor parallelism set to four, and a fix is planned.
The release also completes eagle-harness calibrate --engine vllm on a Qwen3.8 hybrid target. It says calibration tables are keyed by the models resolved by the engine and the generated serve.sh loads them. For TokenSpeed, the draft scores its low-rank candidate head through the engine's linear layer, allowing GEMM dispatch to use a low-latency BF16 kernel. The TokenSpeed engine build included with this release was withdrawn; users are directed to the current install command.
The performance table uses Qwen3.8-27B with its EAGLE-4 draft on four B300 GPUs (TP4), BF16 and temperature zero. Each speedup compares EAGLE-4 throughput with the same engine without speculation in the same allocation, and every EAGLE-4 start passed a twenty-prompt greedy check against no-spec under the fp32 rule. SGLang's high-load recipe reports 7,393.3 tokens/s at concurrency 64 (1.01×), 11,114.3 at 128 (1.00×), 14,332.3 at 256 (1.00×), and 16,197.5 at 512 (0.99×). SGLang's measured recipe reports 733.8 tokens/s at concurrency one (3.35×) and 6,488.2 at 64 (1.38×). The notes caution that a first start measured 0.95× at 64 and 0.97× at 128 while compiling kernels and measuring costs; the high-load rows are warm-cache restarts.
For vLLM, the published measured recipe reaches 8,013.4 tokens/s at concurrency 64 (1.19×); TokenSpeed reaches 6,974.1 at 64 (1.40×). Neither currently switches to plain decoding at high load, and the notes recommend serving them without speculation above concurrency 64. A new install follows the installation guide and installs 0.1.2; 0.1.1 environments must install the listed engine pins and run eagle-harness-check. The SGLang and vLLM 0.1.2 images contain the harness on the same engines; the TokenSpeed image was withdrawn.
What the archive adds
New Runtime's earlier coverage on EAGLE-4 and eagle-harness (2026-10-21) described the source-available drafter and its SGLang, vLLM, and TokenSpeed support. This is a new release with a specific SGLang high-load fallback, additional calibration support, and fresh measurements—not a restatement.
Trend signal
This release supports the idea that a production harness can matter more than a model label: its operational behavior depends on when it drafts and when it falls back. The connection is suggestive rather than decisive, since the notes report a single hardware and model setup. See Harness architecture outlives model choice.
Why now
The release addresses a measurable weakness in the prior SGLang setup: the notes say version 0.1.1 reached 0.74× at concurrency 256 and 0.61× at 512 under the same setup. Version 0.1.2's fallback targets that high-load penalty while retaining low-load speedups.
Connections to other events
New Runtime's earlier coverage of speculative decoding energy measurements found that savings can shrink with batching and may become losses with long drafts or low acceptance. The new release addresses the related operational question of when speculation should be skipped for speed, although its published table measures throughput, not energy. See the earlier energy accounting note.
Why it matters
Speculative decoding is not automatically beneficial at every load. A batch-level fallback gives SGLang operators a way to retain a low-load throughput gain without paying the same drafting cost when concurrency makes plain decoding competitive. The published figures put the trade-off in practical terms: the high-load recipe is almost even with plain decoding at 512 requests, rather than suffering the larger regressions reported for 0.1.1.
This is not yet a general recipe for all deployments. The improvement is tied to four B300 GPUs and a particular Qwen3.8-27B/EAGLE-4 pairing, and it depends on a specific SGLang configuration. Operators using one GPU, keeping radix cache enabled, or running vLLM or TokenSpeed need to follow different limits. The useful takeaway is to benchmark the whole serving configuration across load, rather than treating a low-concurrency speedup as evidence of high-load benefit.
Limits and uncertainty
The release notes report vendor measurements, not an independently reproduced benchmark. The table is limited to one model pair, four B300 GPUs, BF16, and temperature zero; it does not establish performance across hardware, model sizes, or request distributions. High-load rows use warm-cache restarts, while first-start compilation and cost measurement can reduce performance. The notes identify important configuration limits, including disabled radix cache for SGLang fallback and no high-load fallback yet for vLLM or TokenSpeed. No price or hosted-service availability is described.
Skill to build now
If you operate EAGLE-4 on SGLang, benchmark plain decoding and speculative decoding at the concurrency levels your service actually sees. Compare cold starts with warm restarts, verify the radix-cache setting, and record both throughput and the exact engine, checkpoint, and harness versions. For vLLM or TokenSpeed, follow the release's recommendation to serve without speculation above concurrency 64 until their fallback behavior changes.
Reader takeaway
EAGLE-4 0.1.2 makes speculative decoding conditional on whether it wins for a given SGLang batch. The high-load results are close to plain decoding rather than a major speedup, so the release's strongest contribution is load-aware behavior—not a universal throughput claim.