---
type: "post"
stable_id: "post:model-routing-needs-eval-observability-loop"
slug: "model-routing-needs-eval-observability-loop"
title: "Model Routing Is A Control Loop, Not A Static Cost Switch"
description: "Routing saves money only when traces, outcome evals, and fallback policy close the loop between task selection and completed-task quality."
retrieval_nugget: "The useful system observes task class, chosen model, latency, cost, tool trajectory, and final outcome; evaluates routing accuracy and task success; then adjusts policy with a safe fallback. Vendor savings numbers remain claims, not the architecture."
published_at: "2026-08-04"
updated_at: "2026-08-06"
record_date: "2026-08-04"
date_kind: "published_at"
topics: ["model-routing","observability","evals","coding-agents","cost-control"]
entities: ["Not Diamond","groundcover","LangWatch"]
source_urls: ["https://notdiamond-landing.vercel.app/blog/not-diamond-code-intelligent-model-routing-for-coding-agents","https://docs.groundcover.com/capabilities/ai-observability","https://langwatch.ai/docs/ai-gateway/observability","https://langwatch.ai/docs/better-agents/overview"]
source_format: "article"
editorial_timing: {"lane":"regular_hourly","scheduled_at":"2026-08-08T12:00:00+03:00","real_news_delta":"owner-approved primary-source mechanism or merged analysis"}
visual_decision: {"status":"included","reason":"the central mechanism is a flow, loop, architecture, decision, or state transition that benefits from a diagram","reviewed_by":"codex"}
schema_version: "newruntime-agent-readable-v0.2"
status: "published"
visuals: [{"role":"hero","src":"/images/drip/model-routing-needs-eval-observability-loop/model-routing-needs-eval-observability-loop.webp","alt":"A whiteboard feedback loop showing a task router, selected model, trace, outcome evaluation, policy update, and a conservative fallback path.","caption":"New Runtime synthesis from Not Diamond Code, groundcover AI Observability, and LangWatch evaluation."}]
routes: {"html":"https://newruntime.com/posts/model-routing-needs-eval-observability-loop/","markdown":"https://newruntime.com/posts/model-routing-needs-eval-observability-loop.md","json":"https://newruntime.com/posts/model-routing-needs-eval-observability-loop.json"}
---

# Model Routing Is A Control Loop, Not A Static Cost Switch

## Retrieval answer

The useful system observes task class, chosen model, latency, cost, tool trajectory, and final outcome; evaluates routing accuracy and task success; then adjusts policy with a safe fallback. Vendor savings numbers remain claims, not the architecture.

Not Diamond Code argues that coding agents should route different tasks to different models. Groundcover and LangWatch show the missing half of that proposition: every route must emit a trace, preserve tenant and task attribution, and be evaluated against the result. Otherwise the router can lower inference cost while quietly increasing retries, review time, and failed tasks.

A production routing record needs more than model name and token count. It should include task class, selected policy, latency, cost, tool trajectory, fallback events, and a result score tied to a private prompt or scenario set. Routing accuracy and completed-task quality can then be compared across versions instead of inferred from a vendor dashboard.

The decision loop is observe, evaluate, update, and roll back. Start with a conservative baseline, route only bounded task classes, keep an authoritative fallback, and promote a policy when cost per successful task improves without a quality regression. Not Diamond's published savings should remain a vendor claim until reproduced on the actual workload.