{"type":"analysis","id":"analysis:research-looping-the-middle-half-of-an-moe-model-twice-saves-7-18-of-training-compute","slug":"research-looping-the-middle-half-of-an-moe-model-twice-saves-7-18-of-training-compute","title":"Looping the middle half of an MoE model twice saves 7–18% of training compute when budgets are matched, in one proprietary-stack study","description":"SMELT, from Tsinghua, ByteDance Seed, M-A-P and TokenWave.AI, matches per-token FLOPs, parameters and KV cache between looped and plain MoE models. Looping still comes out ahead, but the paper measures no wall-clock cost and releases no code, so for now builders can only treat it as a design hypothesis.","published_at":"2026-10-23T04:00:00.000Z","updated_at":"2026-10-23T04:00:00.000Z","record_date":"2026-10-23","topics":["moe","architecture","design","research"],"source_urls":["https://arxiv.org/html/2609.01343","https://arxiv.org/abs/2609.01343","https://ouro-llm.github.io/","https://arxiv.org/abs/2607.16051"],"language":"en","routes":{"html":"https://newruntime.com/analysis/research-looping-the-middle-half-of-an-moe-model-twice-saves-7-18-of-training-compute/","markdown":"https://newruntime.com/analysis/research-looping-the-middle-half-of-an-moe-model-twice-saves-7-18-of-training-compute.md","json":"https://newruntime.com/analysis/research-looping-the-middle-half-of-an-moe-model-twice-saves-7-18-of-training-compute.json"},"author":{"role":"author","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]},"editorial_provenance":{"desk":"authorial","target_ref":"deep_research:37339327-7642-4e7f-9ca6-c34b21b9ecfc","authorial_article_id":"37339327-7642-4e7f-9ca6-c34b21b9ecfc"},"next_reads":[{"type":"topic","path":"/topics/moe/","url":"https://newruntime.com/topics/moe/","title":"Moe","reason":"Explore the moe topic hub.","media_type":"text/html","markdown_url":"https://newruntime.com/topics/moe.md","json_url":"https://newruntime.com/topics/moe.json"},{"type":"topic","path":"/topics/architecture/","url":"https://newruntime.com/topics/architecture/","title":"Architecture","reason":"Explore the architecture topic hub.","media_type":"text/html","markdown_url":"https://newruntime.com/topics/architecture.md","json_url":"https://newruntime.com/topics/architecture.json"},{"type":"topic","path":"/topics/design/","url":"https://newruntime.com/topics/design/","title":"Design","reason":"Explore the design topic hub.","media_type":"text/html","markdown_url":"https://newruntime.com/topics/design.md","json_url":"https://newruntime.com/topics/design.json"},{"type":"collection","path":"/analysis/","url":"https://newruntime.com/analysis/","title":"Analysis","reason":"Open the latest authorial analysis.","media_type":"text/html","json_url":"https://newruntime.com/analysis.json"},{"type":"collection","path":"/source-ledger.json","url":"https://newruntime.com/source-ledger.json","title":"Source Ledger","reason":"Inspect the public source evidence graph.","media_type":"application/json"}]}