{"type":"analysis","id":"analysis:managed-dynamo-on-hyperpod-removes-the-setup-work-but-still-leaves-you-to-pick","slug":"managed-dynamo-on-hyperpod-removes-the-setup-work-but-still-leaves-you-to-pick","title":"Managed Dynamo on HyperPod removes the setup work but still leaves you to pick prefill/decode topology","description":"AWS's headline result, 2.05× more throughput per GPU from split prefill and decode, comes from a single prompt-heavy request shape. NVIDIA's own recipe for a different model and workload shows aggregated serving ahead on per-GPU throughput. The add-on makes each test cheaper to run, but you still have to choose the topology.","published_at":"2026-11-18T15:00:00.000Z","updated_at":"2026-11-18T15:00:00.000Z","record_date":"2026-11-18","topics":["trends"],"source_urls":["https://aws.amazon.com/blogs/machine-learning/managed-nvidia-dynamo-on-amazon-sagemaker-hyperpod-inference","https://thinkingmachines.ai/model-card/inkling-2-small","https://www.baseten.co/blog/how-to-make-flash-models-even-faster-optimizing-deepseek-v41-flash-1017-fast","https://huggingface.co/unsloth/Ornith-1.5-122B-GGUF"],"language":"en","routes":{"html":"https://newruntime.com/analysis/managed-dynamo-on-hyperpod-removes-the-setup-work-but-still-leaves-you-to-pick/","markdown":"https://newruntime.com/analysis/managed-dynamo-on-hyperpod-removes-the-setup-work-but-still-leaves-you-to-pick.md","json":"https://newruntime.com/analysis/managed-dynamo-on-hyperpod-removes-the-setup-work-but-still-leaves-you-to-pick.json"},"author":{"role":"author","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]},"editorial_provenance":{"desk":"authorial","target_ref":"story_cluster:nvidia-dynamo-support-in-sagemaker-hyperpod-inference-cd89b6f9b7a3","authorial_article_id":"ae10ba21-d070-4fc8-b574-907be01eee61"},"markdown_url":"https://newruntime.com/analysis/managed-dynamo-on-hyperpod-removes-the-setup-work-but-still-leaves-you-to-pick.md","retrieval_nugget":"AWS's headline result, 2.05× more throughput per GPU from split prefill and decode, comes from a single prompt-heavy request shape. NVIDIA's own recipe for a different model and workload shows aggregated serving ahead on per-GPU throughput.","next_reads":[{"type":"collection","path":"/analysis/","url":"https://newruntime.com/analysis/","title":"Analysis","reason":"Open the latest authorial analysis.","media_type":"text/html","json_url":"https://newruntime.com/analysis.json"},{"type":"collection","path":"/sources/","url":"https://newruntime.com/sources/","title":"Sources","reason":"See the original sources this archive cites.","media_type":"text/html"}]}
