What happened
EAGLE-4’s release notes describe the project as combining a draft model with eagle-harness inside an inference engine, adapting speculative decoding to a team’s hardware, batch, and workload. Version 0.1.3 adds native decoding at high load to the SGLang README command, including the one-GPU setup. It also fills the draft key-value cache without prediction work for prompts admitted above the verify limit. The release provides warm-up guidance and says to warm up a server before measuring it.
The release pairs eagle-harness 0.1.3 with SGLang CUDA 0.5.23.dev252+eagle.9, leaves vLLM 0.32.0rc1.dev283+eagle.4 unchanged, and adds TokenSpeed 0.1.1+eagle.8, built from eagle-hq/tokenspeed eagle/v0.1.3. TokenSpeed exposes generic engine hooks; EAGLE-4 policy and tree verification live in the harness plugin. Its supported TokenSpeed path is serve mode, which uses cost tables calibrated once on the target setup. Static mode does not start with the released draft because TokenSpeed 0.1.3 has no sliding-window path for draft trees; the draft uses a 4096-token sliding window. The notes say static-mode support is planned for 0.1.4.
For installation, the notes specify a Python 3.12 environment and instruct users to install eagle-harness and the matching SGLang package together, using the CUDA 13.0 package indexes listed in the release instructions. They also link separate serving guides for SGLang, vLLM, and TokenSpeed. The SGLang, vLLM, and TokenSpeed images are tagged 0.1.3. Users do not need to download anything from the release page itself.
The speed table measures Qwen3.8-27B with the EAGLE-4 draft on NVIDIA B300 GPUs under greedy decoding. Each multiplier compares EAGLE-4 output throughput with the same engine without speculation, averaged over four starts per configuration. On the four-GPU SGLang README command, the reported multiplier is 2.34× at one request and 1.64× at eight, but 0.99× at 512 concurrent requests. The four-GPU SGLang high-load recipe reports 2.90× at one request and 0.99× at 128. The release also gives absolute output-token-per-second results and cautions that throughput does not imply a passing correctness check.
The correctness check compares EAGLE-4 output with the same engine’s output without speculation on the same prompts, judging first differences against an fp32 recomputation. Across four SGLang configurations, 12,089 of 12,096 checked outputs pass. Seven remain outside tolerance: four in the one-GPU SGLang README command at 128 concurrent requests, two in the four-GPU measured recipe at 64, and one in the four-GPU high-load recipe at 512. The vLLM measured recipe passes its release check. The notes say 0.1.4 will compare against plain decoding at the same batch size.
What the archive adds
New Runtime’s earlier coverage described EAGLE-4 as a source-available drafter integrated with SGLang, vLLM, and TokenSpeed, and noted its registration-hook approach. This release is a new event in that thread: it adds a TokenSpeed version, high-load SGLang behavior, and measured correctness limits. See the earlier EAGLE-4 note and the integration note.
Trend signal
This release does not yet fit a pattern New Runtime tracks. It provides evidence about workload-dependent throughput and verification limits, but does not establish a broader shift in agent economics, interoperability, or runtime architecture.
Why now
The release brings a high-load SGLang path and a TokenSpeed serve-mode integration into one version, while documenting measured performance and correctness together. That makes this a useful point for teams already testing EAGLE-4 to reassess their own workloads rather than carry forward expectations from earlier measurements. The notes also identify a planned 0.1.4 change to the correctness comparison method.
Why it matters
Speculative decoding is not a universal throughput multiplier. In these measurements, gains are largest at low concurrency and often shrink toward parity as concurrency rises; some reported cells fall below 1.00×. Teams sizing inference capacity should therefore test their own request mix, engine configuration, GPU count, and concurrency instead of extrapolating from a single headline speedup.
The correctness table is equally important operationally. A high aggregate pass count does not erase the seven SGLang outputs outside the stated tolerance, and the notes distinguish cases where plain decoding on the same build also emits the token at the exact prefix. The release gives infrastructure teams enough detail to reproduce a bounded evaluation, but not a basis for assuming every workload is faster or equivalent. Treating speed and output checks as separate gates is the practical takeaway.
What happens next
- EAGLE-4 0.1.4 will implement the release notes’ planned same-batch-size plain-decoding comparison. (3 months, 70%). Watch for: release notes describing that comparison and reporting its results.
Limits and uncertainty
The published measurements cover Qwen3.8-27B, NVIDIA B300 GPUs, specified commands, and the listed concurrency ranges; they do not establish performance for other models, hardware, or workloads. The release’s correctness check is a defined comparison against plain decoding, not an independent assessment of semantic quality. Seven SGLang outputs remain outside its stated tolerance, and the notes do not claim speculation is faster for every workload. TokenSpeed static mode is unsupported for this draft in 0.1.3; the stated 0.1.4 plan is not a shipped capability.
Skill to build now
This week, reproduce one relevant row of the release’s measurement setup before deciding whether to deploy speculative decoding. Use the same engine, recipe, tensor-parallel size, and running-request cap for the speculative and non-speculative runs; warm up the server first, then record both output tokens per second and the correctness-check result. Include at least one concurrency point representative of your actual traffic.
Reader takeaway
EAGLE-4 0.1.3 makes its inference-engine integrations more usable, but its own tables show why deployment decisions need workload-specific testing. Read the speed multiplier together with the correctness result, and treat the documented high-concurrency limits as part of the release—not as footnotes to its best-case gains.