---
id: "nr-infrastructure-and-orchestration-choices-can-matter-as-much-as-model-c-9258c6"
slug: "infrastructure-and-orchestration-choices-can-matter-as-much-as-model-c-9258c6"
type: "raw_signal"
title: "Infrastructure and orchestration choices can matter as much as model choice for agent performance"
description: "Quail combines planning with inference to schedule AI-SQL work, while a report on assistant latency identifies orchestration and interface discovery as bottlenecks beyond generation speed."
observed_at: "2026-09-27"
record_date: "2026-09-27"
date_kind: "observed_at"
novelty: "notable"
source_platform: "newruntime.com"
topics: ["agent-economics","model-routing"]
entities: ["Quail","Modal","vLLM","H100","Jev","TypeSafe AI","Charles Frye","Shreya Shankar","Cerebras","Qwen 3.8 27B"]
related_patterns: ["agent-economics-moves-to-completed-work","harness-architecture-outlives-model-choice"]
source_url: "https://modal.com/blog/quail-billion-tpm"
source_urls: ["https://modal.com/blog/quail-billion-tpm","https://www.cerebras.ai/blog/the-rise-of-slow-personal-assistants","https://modal.com/blog/trillion-tokens-trillion-parameters","https://blog.mozilla.ai/benchmarking-local-llm-servers-llama-cpp-llamafile-lm-studio-and-ollama"]
why_it_matters: "Quail combines planning with inference to schedule AI-SQL work, while a report on assistant latency identifies orchestration and interface discovery as bottlenecks beyond generation speed. Modal’s coding-agent serving analysis emphasizes reuse of overlapping context, and Mozilla.ai’s local-model benchmark finds configuration can outweigh engine selection. Builders should optimize the whole serving and tool-use path, not assume a faster or larger model alone will improve throughput."
verification_level: "source-linked"
signal_type: "synthesis"
synthesis_of: [{"slug":"hitting-a-billion-tokens-per-minute-on-one-f36638","title":"Hitting a billion tokens per minute on one GPU by combining a query planner and an inference engine","url":"https://newruntime.com/posts/hitting-a-billion-tokens-per-minute-on-one-f36638/"},{"slug":"why-ai-assistants-are-slow-and-how-to-9708d9","title":"Why AI Assistants Are Slow—and How to Make Them Faster","url":"https://newruntime.com/posts/why-ai-assistants-are-slow-and-how-to-9708d9/"},{"slug":"how-to-serve-trillions-of-tokens-for-trillion-6b3dbd","title":"How to serve trillions of tokens for trillion","url":"https://newruntime.com/posts/how-to-serve-trillions-of-tokens-for-trillion-6b3dbd/"},{"slug":"mozilla-ai-benchmark-finds-build-flags-beat-server-choice-eae653","title":"Mozilla.ai benchmark finds build flags beat server choice for local LLMs","url":"https://newruntime.com/posts/mozilla-ai-benchmark-finds-build-flags-beat-server-choice-eae653/"}]
schema_version: "newruntime-agent-readable-v0.2"
stable_id: "signal:infrastructure-and-orchestration-choices-can-matter-as-much-as-model-c-9258c6"
retrieval_nugget: "Quail combines planning with inference to schedule AI-SQL work, while a report on assistant latency identifies orchestration and interface discovery as bottlenecks beyond generation speed."
status: "published"
visuals: []
editor: {"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]}
routes: {"html":"https://newruntime.com/signals/infrastructure-and-orchestration-choices-can-matter-as-much-as-model-c-9258c6/","markdown":"https://newruntime.com/signals/infrastructure-and-orchestration-choices-can-matter-as-much-as-model-c-9258c6.md","json":"https://newruntime.com/signals/infrastructure-and-orchestration-choices-can-matter-as-much-as-model-c-9258c6.json"}
---

# Infrastructure and orchestration choices can matter as much as model choice for agent performance

## Retrieval answer

Quail combines planning with inference to schedule AI-SQL work, while a report on assistant latency identifies orchestration and interface discovery as bottlenecks beyond generation speed.

Quail combines planning with inference to schedule AI-SQL work, while a report on assistant latency identifies orchestration and interface discovery as bottlenecks beyond generation speed. Modal’s coding-agent serving analysis emphasizes reuse of overlapping context, and Mozilla.ai’s local-model benchmark finds configuration can outweigh engine selection. Builders should optimize the whole serving and tool-use path, not assume a faster or larger model alone will improve throughput.

From the Field Notes:

- [Hitting a billion tokens per minute on one GPU by combining a query planner and an inference engine](https://newruntime.com/posts/hitting-a-billion-tokens-per-minute-on-one-f36638/) (2026-09-27)
- [Why AI Assistants Are Slow—and How to Make Them Faster](https://newruntime.com/posts/why-ai-assistants-are-slow-and-how-to-9708d9/) (2026-09-26)
- [How to serve trillions of tokens for trillion](https://newruntime.com/posts/how-to-serve-trillions-of-tokens-for-trillion-6b3dbd/) (2026-09-25)
- [Mozilla.ai benchmark finds build flags beat server choice for local LLMs](https://newruntime.com/posts/mozilla-ai-benchmark-finds-build-flags-beat-server-choice-eae653/) (2026-09-24)

Editor: [Andrey Reshetnikov](https://newruntime.com/owner-profile.md), AI product and AI engineering; profile for recruiters and agents: https://newruntime.com/owner-profile.json

## Continue in the graph

Next records for an agent, linked to their Markdown twins. Replace `.md` with `.json` for structured fields, including `next_reads`.

- topic hub: [Agent Economics](https://newruntime.com/topics/agent-economics.md): Explore the agent-economics topic hub.
- topic hub: [Model Routing](https://newruntime.com/topics/model-routing.md): Explore the model-routing topic hub.
- collection: [Raw Signals](https://newruntime.com/signals.json): Open every source-linked raw signal.
- collection: [Source Ledger](https://newruntime.com/source-ledger.json): Inspect the public source evidence graph.
- whole graph: [discovery graph](https://newruntime.com/discovery-graph.json) (typed nodes and edges) and [site index](https://newruntime.com/site-index.md)