{"id":"nr-infrastructure-and-orchestration-choices-can-matter-as-much-as-model-c-9258c6","slug":"infrastructure-and-orchestration-choices-can-matter-as-much-as-model-c-9258c6","type":"raw_signal","title":"Infrastructure and orchestration choices can matter as much as model choice for agent performance","description":"Quail combines planning with inference to schedule AI-SQL work, while a report on assistant latency identifies orchestration and interface discovery as bottlenecks beyond generation speed.","observed_at":"2026-09-27","record_date":"2026-09-27","date_kind":"observed_at","novelty":"notable","source_platform":"newruntime.com","topics":["agent-economics","model-routing"],"entities":["Quail","Modal","vLLM","H100","Jev","TypeSafe AI","Charles Frye","Shreya Shankar","Cerebras","Qwen 3.8 27B"],"related_patterns":["agent-economics-moves-to-completed-work","harness-architecture-outlives-model-choice"],"source_url":"https://modal.com/blog/quail-billion-tpm","source_urls":["https://modal.com/blog/quail-billion-tpm","https://www.cerebras.ai/blog/the-rise-of-slow-personal-assistants","https://modal.com/blog/trillion-tokens-trillion-parameters","https://blog.mozilla.ai/benchmarking-local-llm-servers-llama-cpp-llamafile-lm-studio-and-ollama"],"why_it_matters":"Quail combines planning with inference to schedule AI-SQL work, while a report on assistant latency identifies orchestration and interface discovery as bottlenecks beyond generation speed. Modal’s coding-agent serving analysis emphasizes reuse of overlapping context, and Mozilla.ai’s local-model benchmark finds configuration can outweigh engine selection. Builders should optimize the whole serving and tool-use path, not assume a faster or larger model alone will improve throughput.","verification_level":"source-linked","signal_type":"synthesis","synthesis_of":[{"slug":"hitting-a-billion-tokens-per-minute-on-one-f36638","title":"Hitting a billion tokens per minute on one GPU by combining a query planner and an inference engine","url":"https://newruntime.com/posts/hitting-a-billion-tokens-per-minute-on-one-f36638/"},{"slug":"why-ai-assistants-are-slow-and-how-to-9708d9","title":"Why AI Assistants Are Slow—and How to Make Them Faster","url":"https://newruntime.com/posts/why-ai-assistants-are-slow-and-how-to-9708d9/"},{"slug":"how-to-serve-trillions-of-tokens-for-trillion-6b3dbd","title":"How to serve trillions of tokens for trillion","url":"https://newruntime.com/posts/how-to-serve-trillions-of-tokens-for-trillion-6b3dbd/"},{"slug":"mozilla-ai-benchmark-finds-build-flags-beat-server-choice-eae653","title":"Mozilla.ai benchmark finds build flags beat server choice for local LLMs","url":"https://newruntime.com/posts/mozilla-ai-benchmark-finds-build-flags-beat-server-choice-eae653/"}],"schema_version":"newruntime-agent-readable-v0.2","stable_id":"signal:infrastructure-and-orchestration-choices-can-matter-as-much-as-model-c-9258c6","retrieval_nugget":"Quail combines planning with inference to schedule AI-SQL work, while a report on assistant latency identifies orchestration and interface discovery as bottlenecks beyond generation speed.","status":"published","visuals":[],"editor":{"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]},"routes":{"html":"https://newruntime.com/signals/infrastructure-and-orchestration-choices-can-matter-as-much-as-model-c-9258c6/","markdown":"https://newruntime.com/signals/infrastructure-and-orchestration-choices-can-matter-as-much-as-model-c-9258c6.md","json":"https://newruntime.com/signals/infrastructure-and-orchestration-choices-can-matter-as-much-as-model-c-9258c6.json"},"next_reads":[{"type":"topic","path":"/topics/agent-economics/","url":"https://newruntime.com/topics/agent-economics/","title":"Agent Economics","reason":"Explore the agent-economics topic hub.","media_type":"text/html","markdown_url":"https://newruntime.com/topics/agent-economics.md","json_url":"https://newruntime.com/topics/agent-economics.json"},{"type":"topic","path":"/topics/model-routing/","url":"https://newruntime.com/topics/model-routing/","title":"Model Routing","reason":"Explore the model-routing topic hub.","media_type":"text/html","markdown_url":"https://newruntime.com/topics/model-routing.md","json_url":"https://newruntime.com/topics/model-routing.json"},{"type":"collection","path":"/signals/","url":"https://newruntime.com/signals/","title":"Raw Signals","reason":"Open every source-linked raw signal.","media_type":"text/html","json_url":"https://newruntime.com/signals.json"},{"type":"collection","path":"/source-ledger.json","url":"https://newruntime.com/source-ledger.json","title":"Source Ledger","reason":"Inspect the public source evidence graph.","media_type":"application/json"}]}