{"type":"post","id":"inferssd-streams-moe-expert-weights-from-nvme-running-c0923f","slug":"inferssd-streams-moe-expert-weights-from-nvme-running-c0923f","title":"InferSSD streams MoE expert weights from NVMe, running 700B+ models on consumer hardware.","description":"InferSSD is an open-source llama.cpp-based runtime that streams mixture-of-experts model weights from SSD, aiming to run models larger than a machine’s RAM or VRAM.","retrieval_nugget":"InferSSD is an open-source llama.cpp-based runtime that streams mixture-of-experts model weights from SSD, aiming to run models larger than a machine’s RAM or VRAM. Its repository reports tests of models up to 816 billion total parameters and publishes comparisons across several hardware setups, making local inference trade-offs more concrete for builders.","published_at":"2026-11-14","updated_at":"2026-11-14","record_date":"2026-11-14","date_kind":"published_at","topics":["models"],"entities":["InferSSD","wici-ai","llama.cpp","DeepSeek-V4.1-Flash-1017","Muse-Spark-Open","Qwen3.6-35B-A3B","Qwen3.8 Flash Next"],"related_patterns":[],"novelty_label":"new_fact","what_changed":"InferSSD adds another NVMe-streamed MoE runtime, claiming it can run models larger than available RAM or VRAM, including 700B-plus models, on consumer hardware.","prior_refs":["https://newruntime.com/posts/betelgeuse-llama-cpp-fork-pages-moe-experts-from-fb5996/","https://newruntime.com/posts/slotstream-runs-a-125b-moe-model-on-16-1dfd4c/"],"source_url":"https://github.com/wici-ai/inferssd","source_urls":["https://github.com/wici-ai/inferssd"],"schema_version":"newruntime-agent-readable-v0.2","stable_id":"post:inferssd-streams-moe-expert-weights-from-nvme-running-c0923f","status":"published","visuals":[],"editor":{"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]},"routes":{"html":"https://newruntime.com/posts/inferssd-streams-moe-expert-weights-from-nvme-running-c0923f/","markdown":"https://newruntime.com/posts/inferssd-streams-moe-expert-weights-from-nvme-running-c0923f.md","json":"https://newruntime.com/posts/inferssd-streams-moe-expert-weights-from-nvme-running-c0923f.json"},"markdown_url":"https://newruntime.com/posts/inferssd-streams-moe-expert-weights-from-nvme-running-c0923f.md","next_reads":[{"type":"topic","path":"/topics/models/","url":"https://newruntime.com/topics/models/","title":"Models","reason":"Explore the models topic hub.","media_type":"text/html","markdown_url":"https://newruntime.com/topics/models.md","json_url":"https://newruntime.com/topics/models.json"},{"type":"collection","path":"/posts/","url":"https://newruntime.com/posts/","title":"Field Notes","reason":"Open the latest editorial analysis.","media_type":"text/html","json_url":"https://newruntime.com/posts.json"},{"type":"collection","path":"/sources/","url":"https://newruntime.com/sources/","title":"Sources","reason":"See the original sources this archive cites.","media_type":"text/html"}]}