{"type":"post","id":"building-an-automated-improvement-harness-for-an-ai-74a2f3","slug":"building-an-automated-improvement-harness-for-an-ai-74a2f3","title":"Building an Automated Improvement Harness for an AI Support Agent Engineering Gen AI LLM ML Infrastructure","description":"DoorDash says it built an automated improvement harness around its internal AI support agent, using resolution-based evaluation to find and address recurring failures.","retrieval_nugget":"DoorDash says it built an automated improvement harness around its internal AI support agent, using resolution-based evaluation to find and address recurring failures. In the first couple of months, the company reports a greater-than-200% increase in full resolution rate and roughly 100 knowledge-base corrections and patches—a concrete example of turning support usage into a governed improvement loop.","published_at":"2026-11-11","updated_at":"2026-11-11","record_date":"2026-11-11","date_kind":"published_at","topics":["agent-harness","evals","harness-engineering"],"entities":["DoorDash","Flux"],"related_patterns":["harness-architecture-outlives-model-choice","verification-bandwidth-is-the-scarce-resource"],"novelty_label":"new_fact","what_changed":"DoorDash describes an automated, expert-aligned harness that benchmarks its AI support agent and turns recurring unresolved cases into targeted improvements.","prior_refs":[],"source_url":"https://careersatdoordash.com/blog/building-an-automated-improvement-harness-for-an-ai-support-agent","source_urls":["https://careersatdoordash.com/blog/building-an-automated-improvement-harness-for-an-ai-support-agent"],"schema_version":"newruntime-agent-readable-v0.2","stable_id":"post:building-an-automated-improvement-harness-for-an-ai-74a2f3","status":"published","visuals":[],"editor":{"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]},"routes":{"html":"https://newruntime.com/posts/building-an-automated-improvement-harness-for-an-ai-74a2f3/","markdown":"https://newruntime.com/posts/building-an-automated-improvement-harness-for-an-ai-74a2f3.md","json":"https://newruntime.com/posts/building-an-automated-improvement-harness-for-an-ai-74a2f3.json"},"markdown_url":"https://newruntime.com/posts/building-an-automated-improvement-harness-for-an-ai-74a2f3.md","next_reads":[{"type":"topic","path":"/topics/agent-harness/","url":"https://newruntime.com/topics/agent-harness/","title":"Agent Harness","reason":"Explore the agent-harness topic hub.","media_type":"text/html","markdown_url":"https://newruntime.com/topics/agent-harness.md","json_url":"https://newruntime.com/topics/agent-harness.json"},{"type":"topic","path":"/topics/evals/","url":"https://newruntime.com/topics/evals/","title":"Evals","reason":"Explore the evals topic hub.","media_type":"text/html","markdown_url":"https://newruntime.com/topics/evals.md","json_url":"https://newruntime.com/topics/evals.json"},{"type":"topic","path":"/topics/harness-engineering/","url":"https://newruntime.com/topics/harness-engineering/","title":"Harness Engineering","reason":"Explore the harness-engineering topic hub.","media_type":"text/html","markdown_url":"https://newruntime.com/topics/harness-engineering.md","json_url":"https://newruntime.com/topics/harness-engineering.json"},{"type":"collection","path":"/posts/","url":"https://newruntime.com/posts/","title":"Field Notes","reason":"Open the latest editorial analysis.","media_type":"text/html","json_url":"https://newruntime.com/posts.json"},{"type":"collection","path":"/sources/","url":"https://newruntime.com/sources/","title":"Sources","reason":"See the original sources this archive cites.","media_type":"text/html"}]}