{"type":"raw_signal","id":"push-based-kv-transfer-to-improve-ttft-and-4eb450","slug":"push-based-kv-transfer-to-improve-ttft-and-4eb450","title":"Push-Based KV Transfer to Improve TTFT and TPOT for Disaggregated Inference in vLLM (15 minute read)","description":"vLLM has introduced a push-based NIXL connector for transferring KV cache in disaggregated prefill/decode inference.","retrieval_nugget":"vLLM has introduced a push-based NIXL connector for transferring KV cache in disaggregated prefill/decode inference.","observed_at":"2026-10-15","record_date":"2026-10-15","date_kind":"observed_at","why_it_matters":"The vLLM post reports results from a 1P1D Qwen3-32B benchmark on Amazon EC2 P5en instances connected with EFA: up to 3× better time to first token (TTFT) and up to 30% better time per output token (TPOT) than pull mode. The authors note that results depend on workload and configuration. They say the feature was built and validated on AWS Trainium clusters and contributed to vLLM as accelerator-agn","novelty":"notable","verification_level":"source-linked","signal_type":"research","source_platform":"vllm.ai","topics":[],"entities":["vLLM","NIXL","Qwen3-32B","Amazon Web Services","Amazon EC2 P5en","AWS Elastic Fabric Adapter"],"related_patterns":["harness-architecture-outlives-model-choice"],"source_url":"https://vllm.ai/blog/2026-10-14-nixl-push-connector","source_urls":["https://vllm.ai/blog/2026-10-14-nixl-push-connector"],"schema_version":"newruntime-agent-readable-v0.2","stable_id":"signal:push-based-kv-transfer-to-improve-ttft-and-4eb450","status":"published","visuals":[],"editor":{"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]},"routes":{"html":"https://newruntime.com/signals/push-based-kv-transfer-to-improve-ttft-and-4eb450/","markdown":"https://newruntime.com/signals/push-based-kv-transfer-to-improve-ttft-and-4eb450.md","json":"https://newruntime.com/signals/push-based-kv-transfer-to-improve-ttft-and-4eb450.json"},"next_reads":[{"type":"collection","path":"/signals/","url":"https://newruntime.com/signals/","title":"Raw Signals","reason":"Open every source-linked raw signal.","media_type":"text/html","json_url":"https://newruntime.com/signals.json"},{"type":"collection","path":"/source-ledger.json","url":"https://newruntime.com/source-ledger.json","title":"Source Ledger","reason":"Inspect the public source evidence graph.","media_type":"application/json"}]}