---
id: "nr-agent-reliability-gains-are-coming-from-the-harness-and-learned-experi-89264d"
slug: "agent-reliability-gains-are-coming-from-the-harness-and-learned-experi-89264d"
type: "raw_signal"
title: "Agent reliability gains are coming from the harness and learned experience, not bigger models"
description: "A behavioral harness took a cheaper coding model from 3 of 10 successful runs to 10 of 10, at 33% less per run than the larger model."
observed_at: "2026-10-23"
record_date: "2026-10-23"
date_kind: "observed_at"
novelty: "notable"
source_platform: "newruntime.com"
topics: []
entities: ["Naresh Jain","GPT-6 Luna","GPT-6 Sol","Specmatic","Codex","IBM Research","ALTK-Evolve","τ²-bench","GPT-4.1","GPT-5.5"]
related_patterns: ["harness-architecture-outlives-model-choice","verification-bandwidth-is-the-scarce-resource"]
source_url: "https://martinfowler.com/articles/are-harnesses-worth-the-cost.html"
source_urls: ["https://martinfowler.com/articles/are-harnesses-worth-the-cost.html","https://huggingface.co/blog/ibm-research/altk-evolve-bc","https://kiro.dev/blog/how-kiro-fixes-itself","https://www.cerebras.ai/blog/a-smarter-model-won-t-save-your-bad-harness","https://aistudio.google.com/learn/agentic-video-understanding-with-gemini"]
why_it_matters: "A behavioral harness took a cheaper coding model from 3 of 10 successful runs to 10 of 10, at 33% less per run than the larger model. Learned memory put an older model at the top of a customer-service leaderboard without changing its weights. Amazon's Kiro now mines its own runs to propose prompt and tool fixes. Before paying for a model upgrade, builders should invest in loop design, memory and self-tuning harnesses."
verification_level: "source-linked"
signal_type: "synthesis"
synthesis_of: [{"slug":"cheap-but-unreliable-smaller-ai-models-are-tempting-057529","title":"Cheap, but unreliable. Smaller AI models are tempting for coding tasks because they cost less, and teams rarely choose a model on raw intelligence alone.","url":"https://newruntime.com/posts/cheap-but-unreliable-smaller-ai-models-are-tempting-057529/"},{"slug":"altk-evolve-how-learned-memory-tops-the-2-fae5de","title":"ALTK-Evolve: How learned memory tops the τ²-bench leaderboard on customer service tasks without a frontier model","url":"https://newruntime.com/posts/altk-evolve-how-learned-memory-tops-the-2-fae5de/"},{"slug":"on-tuesday-amazon-unveiled-kiro-trace-an-internal-a63578","title":"On Tuesday, Amazon unveiled Kiro-trace , an internal tracing system that improves Kiro, Amazon's AI coding agent, from its own runs.","url":"https://newruntime.com/posts/on-tuesday-amazon-unveiled-kiro-trace-an-internal-a63578/"},{"slug":"a-smarter-model-won-t-save-your-bad-36f226","title":"A Smarter Model Won’t Save Your Bad Harness","url":"https://newruntime.com/posts/a-smarter-model-won-t-save-your-bad-36f226/"},{"slug":"google-launched-agentic-video-understanding-for-gemini-flash-ad81da","title":"Google launched agentic video understanding for Gemini Flash models, cutting long-form video costs up to 66%.","url":"https://newruntime.com/posts/google-launched-agentic-video-understanding-for-gemini-flash-ad81da/"}]
schema_version: "newruntime-agent-readable-v0.2"
stable_id: "signal:agent-reliability-gains-are-coming-from-the-harness-and-learned-experi-89264d"
retrieval_nugget: "A behavioral harness took a cheaper coding model from 3 of 10 successful runs to 10 of 10, at 33% less per run than the larger model."
status: "published"
visuals: []
editor: {"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]}
routes: {"html":"https://newruntime.com/signals/agent-reliability-gains-are-coming-from-the-harness-and-learned-experi-89264d/","markdown":"https://newruntime.com/signals/agent-reliability-gains-are-coming-from-the-harness-and-learned-experi-89264d.md","json":"https://newruntime.com/signals/agent-reliability-gains-are-coming-from-the-harness-and-learned-experi-89264d.json"}
---

# Agent reliability gains are coming from the harness and learned experience, not bigger models

## Retrieval answer

A behavioral harness took a cheaper coding model from 3 of 10 successful runs to 10 of 10, at 33% less per run than the larger model.

A behavioral harness took a cheaper coding model from 3 of 10 successful runs to 10 of 10, at 33% less per run than the larger model. Learned memory put an older model at the top of a customer-service leaderboard without changing its weights. Amazon's Kiro now mines its own runs to propose prompt and tool fixes. Before paying for a model upgrade, builders should invest in loop design, memory and self-tuning harnesses.

From the Field Notes:

- [Cheap, but unreliable. Smaller AI models are tempting for coding tasks because they cost less, and teams rarely choose a model on raw intelligence alone.](https://newruntime.com/posts/cheap-but-unreliable-smaller-ai-models-are-tempting-057529/) (2026-10-22)
- [ALTK-Evolve: How learned memory tops the τ²-bench leaderboard on customer service tasks without a frontier model](https://newruntime.com/posts/altk-evolve-how-learned-memory-tops-the-2-fae5de/) (2026-10-22)
- [On Tuesday, Amazon unveiled Kiro-trace , an internal tracing system that improves Kiro, Amazon's AI coding agent, from its own runs.](https://newruntime.com/posts/on-tuesday-amazon-unveiled-kiro-trace-an-internal-a63578/) (2026-10-22)
- [A Smarter Model Won’t Save Your Bad Harness](https://newruntime.com/posts/a-smarter-model-won-t-save-your-bad-36f226/) (2026-10-22)
- [Google launched agentic video understanding for Gemini Flash models, cutting long-form video costs up to 66%.](https://newruntime.com/posts/google-launched-agentic-video-understanding-for-gemini-flash-ad81da/) (2026-10-22)

Editor: [Andrey Reshetnikov](https://newruntime.com/owner-profile.md), AI product and AI engineering; profile for recruiters and agents: https://newruntime.com/owner-profile.json

## Continue in the graph

Next records for an agent, linked to their Markdown twins. Replace `.md` with `.json` for structured fields, including `next_reads`.

- collection: [Raw Signals](https://newruntime.com/signals.json): Open every source-linked raw signal.
- collection: [Source Ledger](https://newruntime.com/source-ledger.json): Inspect the public source evidence graph.
- whole graph: [discovery graph](https://newruntime.com/discovery-graph.json) (typed nodes and edges) and [site index](https://newruntime.com/site-index.md)