---
type: "raw_signal"
id: "push-based-kv-transfer-to-improve-ttft-and-4eb450"
slug: "push-based-kv-transfer-to-improve-ttft-and-4eb450"
title: "Push-Based KV Transfer to Improve TTFT and TPOT for Disaggregated Inference in vLLM (15 minute read)"
description: "vLLM has introduced a push-based NIXL connector for transferring KV cache in disaggregated prefill/decode inference."
retrieval_nugget: "vLLM has introduced a push-based NIXL connector for transferring KV cache in disaggregated prefill/decode inference."
observed_at: "2026-10-15"
record_date: "2026-10-15"
date_kind: "observed_at"
why_it_matters: "The vLLM post reports results from a 1P1D Qwen3-32B benchmark on Amazon EC2 P5en instances connected with EFA: up to 3× better time to first token (TTFT) and up to 30% better time per output token (TPOT) than pull mode. The authors note that results depend on workload and configuration. They say the feature was built and validated on AWS Trainium clusters and contributed to vLLM as accelerator-agn"
novelty: "notable"
verification_level: "source-linked"
signal_type: "research"
source_platform: "vllm.ai"
topics: []
entities: ["vLLM","NIXL","Qwen3-32B","Amazon Web Services","Amazon EC2 P5en","AWS Elastic Fabric Adapter"]
related_patterns: ["harness-architecture-outlives-model-choice"]
source_url: "https://vllm.ai/blog/2026-10-14-nixl-push-connector"
source_urls: ["https://vllm.ai/blog/2026-10-14-nixl-push-connector"]
schema_version: "newruntime-agent-readable-v0.2"
stable_id: "signal:push-based-kv-transfer-to-improve-ttft-and-4eb450"
status: "published"
visuals: []
editor: {"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]}
routes: {"html":"https://newruntime.com/signals/push-based-kv-transfer-to-improve-ttft-and-4eb450/","markdown":"https://newruntime.com/signals/push-based-kv-transfer-to-improve-ttft-and-4eb450.md","json":"https://newruntime.com/signals/push-based-kv-transfer-to-improve-ttft-and-4eb450.json"}
---

# Push-Based KV Transfer to Improve TTFT and TPOT for Disaggregated Inference in vLLM (15 minute read)

## Retrieval answer

vLLM has introduced a push-based NIXL connector for transferring KV cache in disaggregated prefill/decode inference.

vLLM has introduced a push-based NIXL connector for transferring KV cache in disaggregated prefill/decode inference. In the default pull flow, the decode instance waits for routing metadata, allocates destination blocks, prepares the transfer, and initiates a read after prefill completes. The new flow registers decode-side destination blocks in advance; once prefill and registration are both ready, the prefill side issues a NIXL write, with setup handled by a background thread.

The vLLM post reports results from a 1P1D Qwen3-32B benchmark on Amazon EC2 P5en instances connected with EFA: up to 3× better time to first token (TTFT) and up to 30% better time per output token (TPOT) than pull mode. The authors note that results depend on workload and configuration. They say the feature was built and validated on AWS Trainium clusters and contributed to vLLM as accelerator-agnostic. https://vllm.ai/blog/2026-10-14-nixl-push-connector

Editor: [Andrey Reshetnikov](https://newruntime.com/owner-profile.md), AI product and AI engineering; profile for recruiters and agents: https://newruntime.com/owner-profile.json

## Continue in the graph

Next records for an agent, linked to their Markdown twins. Replace `.md` with `.json` for structured fields, including `next_reads`.

- collection: [Raw Signals](https://newruntime.com/signals.json): Open every source-linked raw signal.
- collection: [Source Ledger](https://newruntime.com/source-ledger.json): Inspect the public source evidence graph.
- whole graph: [discovery graph](https://newruntime.com/discovery-graph.json) (typed nodes and edges) and [site index](https://newruntime.com/site-index.md)