{"type":"post","id":"nvlink-sharded-engram-tables-each-prefill-instance-previously-907257","slug":"nvlink-sharded-engram-tables-each-prefill-instance-previously-907257","title":"NVLink-sharded Engram tables: Each prefill instance previously stored two 94.4 GiB n-gram tables in host memory, consuming 378 GiB per tray.","description":"A vLLM pull request proposes splitting DeepSeek-V4.1 Engram GPU tables across engines in an NVLink domain, reducing the table footprint per GPU and freeing host memory.","retrieval_nugget":"A vLLM pull request proposes splitting DeepSeek-V4.1 Engram GPU tables across engines in an NVLink domain, reducing the table footprint per GPU and freeing host memory. It also changes lookup tiling for large workloads; reported tests show faster steps in selected prefill configurations, while preserving exact lookup outputs.","published_at":"2026-11-13","updated_at":"2026-11-13","record_date":"2026-11-13","date_kind":"published_at","topics":["memory"],"entities":["vLLM","Engram","DeepSeek-V4.1-Flash","NVIDIA GB300"],"related_patterns":["agent-economics-moves-to-completed-work"],"novelty_label":"new_fact","what_changed":"vLLM adds NVLink-domain Engram table sharding across GPUs, freeing 188.8 GiB of host memory per engine for DeepSeek-V4.1-Flash, alongside larger lookup tiles.","prior_refs":["https://newruntime.com/posts/vllm-blog-serving-deepseek-v4-1-flash-at-073f14/","https://newruntime.com/posts/full-engineering-writeup-covers-dep2-prefill-wide-ep-b2079d/"],"source_url":"https://github.com/vllm-project/vllm/pull/66552","source_urls":["https://github.com/vllm-project/vllm/pull/66552"],"schema_version":"newruntime-agent-readable-v0.2","stable_id":"post:nvlink-sharded-engram-tables-each-prefill-instance-previously-907257","status":"published","visuals":[],"editor":{"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]},"routes":{"html":"https://newruntime.com/posts/nvlink-sharded-engram-tables-each-prefill-instance-previously-907257/","markdown":"https://newruntime.com/posts/nvlink-sharded-engram-tables-each-prefill-instance-previously-907257.md","json":"https://newruntime.com/posts/nvlink-sharded-engram-tables-each-prefill-instance-previously-907257.json"},"markdown_url":"https://newruntime.com/posts/nvlink-sharded-engram-tables-each-prefill-instance-previously-907257.md","next_reads":[{"type":"topic","path":"/topics/memory/","url":"https://newruntime.com/topics/memory/","title":"Memory","reason":"Explore the memory topic hub.","media_type":"text/html","markdown_url":"https://newruntime.com/topics/memory.md","json_url":"https://newruntime.com/topics/memory.json"},{"type":"collection","path":"/posts/","url":"https://newruntime.com/posts/","title":"Field Notes","reason":"Open the latest editorial analysis.","media_type":"text/html","json_url":"https://newruntime.com/posts.json"},{"type":"collection","path":"/sources/","url":"https://newruntime.com/sources/","title":"Sources","reason":"See the original sources this archive cites.","media_type":"text/html"}]}