{"schema_version":"newruntime-agent-readable-v0.2","type":"post","stable_id":"post:agent-harness-optimization-outer-loop","slug":"agent-harness-optimization-outer-loop","title":"Agent Harness Optimization Is Becoming an Outer-Loop Discipline","description":"The next useful automation target is not another agent response. It is the controlled loop that changes prompts, tools, context, and routing, then keeps only improvements that survive evaluation.","retrieval_nugget":"The next useful automation target is not another agent response. It is the controlled loop that changes prompts, tools, context, and routing, then keeps only improvements that survive evaluation. Most agent improvement is still manual: edit a prompt, change the available tools, adjust context, rerun a few examples, and decide by feel whether the system became better.","status":"published","published_at":"2026-07-24","updated_at":"2026-07-24","record_date":"2026-07-24","date_kind":"published_at","topics":["agent-harness","evals","autoresearch","optimization"],"source_urls":["https://x.com/_avichawla/status/2080577991990751611","https://x.com/akshay_pachaar/status/2049916107923034300","https://github.com/karpathy/autoresearch","https://arxiv.org/abs/2602.03786","https://github.com/FoundationAgents/AOrchestra"],"visuals":[{"id":"harness-optimization-outer-loop","kind":"editorial-diagram","role":"hero","src":"https://newruntime.com/images/posts/harness-optimization-outer-loop.webp","alt":"A controlled five-stage optimization loop surrounds a locked evaluator while bounded harness changes are tested, scored, kept or discarded, and logged.","caption":"A trustworthy outer loop may mutate the harness, but fixtures, scoring, comparison, rollback, and promotion remain deterministic.","credit":"New Runtime synthesis from autoresearch and agent-system optimization research","source_url":"https://github.com/karpathy/autoresearch","generated_with":"gemini-3.1-flash-image","width":1600,"height":900,"legend":[{"label":"Bound the mutation","description":"Change one versioned prompt, tool contract, context rule, route, or model setting at a time."},{"label":"Freeze evaluation","description":"Run the candidate against unchanged fixtures, time budgets, and protected cases."},{"label":"Score broadly","description":"Compare quality, safety, latency, cost, retries, regressions, and required escalation."},{"label":"Decide","description":"Keep, discard, or record a crash without letting the optimizer redefine success."},{"label":"Audit","description":"Preserve every candidate and result before another bounded experiment begins."}]}],"routes":{"html":"https://newruntime.com/posts/agent-harness-optimization-outer-loop/","markdown":"https://newruntime.com/posts/agent-harness-optimization-outer-loop.md","json":"https://newruntime.com/posts/agent-harness-optimization-outer-loop.json"},"source_format":"markdown","next_reads":[{"type":"topic","path":"/topics/agent-harness/","reason":"Explore the agent harness topic hub.","url":"https://newruntime.com/topics/agent-harness/","title":"Agent Harness - New Runtime","media_type":"text/html"},{"type":"topic","path":"/topics/evals/","reason":"Explore the evals topic hub.","url":"https://newruntime.com/topics/evals/","title":"Agent evals - New Runtime","media_type":"text/html"},{"type":"related_material","path":"/posts/agentic-sdlc-software-factory-loop/","reason":"Shares evals.","url":"https://newruntime.com/posts/agentic-sdlc-software-factory-loop/","title":"A Software Factory Connects Agents Through Verified Outcomes","media_type":"text/html"},{"type":"related_material","path":"/posts/cerebras-moe-router-gradient-null-expert/","reason":"Shares evals.","url":"https://newruntime.com/posts/cerebras-moe-router-gradient-null-expert/","title":"A Balanced MoE Router Can Still Be Functionally Dead","media_type":"text/html"},{"type":"related_material","path":"/posts/claude-code-auto-mode-action-gate/","reason":"Shares evals.","url":"https://newruntime.com/posts/claude-code-auto-mode-action-gate/","title":"Claude Code Auto Mode Gates Actions Instead Of Explanations","media_type":"text/html"}]}