---
schema_version: "newruntime-topic-hub-v0.1"
type: "topic_hub"
slug: "evals"
title: "Agent evals - New Runtime"
description: "A record of evals moving from model benchmarks toward behavior, workflow completion, skill reliability, and review burden."
answer: ["Agent evals need to measure behavior in a workflow, not only answer quality in a prompt.","The relevant unit is completed work under constraints: tools, files, permissions, time, cost, and review outcome.","A good eval makes regressions visible before the agent is trusted with larger work."]
search_intents: ["agent evals","AI agent evaluation","LLM workflow benchmarks"]
status: "featured"
last_updated: "2026-07-16"
counts: {"total":13,"signals":11,"patterns":2,"posts":0,"atlas":0,"sources":22}
routes: {"html":"https://newruntime.com/topics/evals/","markdown":"https://newruntime.com/topics/evals.md","json":"https://newruntime.com/topics/evals.json"}
top_sources: ["https://addyo.substack.com/p/own-the-outer-loop","https://addyosmani.com/blog/new-sdlc-vibe-coding","https://anthropic.com/research/AI-assistance-coding-skills","https://arxiv.org/abs/2602.12670","https://arxiv.org/abs/2603.29919","https://arxiv.org/abs/2605.24050","https://databricks.com/blog/benchmarking-coding-agents-databricks-multi-million-line-codebase","https://dbreunig.com/2026/06/22/the-problem-is-prompt-debt.html","https://developers.openai.com/api/docs/guides/latest-model","https://github.com/benchflow-ai/skillsbench","https://github.com/OpenDataBox/MemoryData","https://github.com/shadcn/improve"]
---

# Agent evals - New Runtime

A record of evals moving from model benchmarks toward behavior, workflow completion, skill reliability, and review burden.

## Short Answer

- Agent evals need to measure behavior in a workflow, not only answer quality in a prompt.
- The relevant unit is completed work under constraints: tools, files, permissions, time, cost, and review outcome.
- A good eval makes regressions visible before the agent is trusted with larger work.

## Patterns

- [Skills become a portable capability layer](https://newruntime.com/patterns/skills-become-portable-capability-layer/): Agent skills are emerging as a portable capability layer, but their value depends on progressive disclosure, provenance, security review, and behavioral evaluation.
- [Verification bandwidth is the scarce engineering resource](https://newruntime.com/patterns/verification-bandwidth-is-the-scarce-resource/): The primary bottleneck in agentic software delivery is moving from code production to the human and machine capacity required to verify it.

## Field Notes


## Recent Raw Signals

- 2026-07-16: [Agent skills need behavioral evals, not prose review](https://newruntime.com/signals/agent-skills-need-behavioral-evals/)
- 2026-07-16: [Verification capacity becomes the coding-agent bottleneck](https://newruntime.com/signals/verification-capacity-coding-agent-bottleneck/)
- 2026-07-14: [AI product feedback needs behavioral traces](https://newruntime.com/signals/ai-feedback-needs-behavioral-traces/)
- 2026-07-12: [Databricks benchmarks coding agents on its own codebase](https://newruntime.com/signals/databricks-benchmarks-real-coding-agent-economics/)
- 2026-07-10: [Newer agent models can reward shorter prompts](https://newruntime.com/signals/gpt-5-6-shorter-prompts/)
- 2026-07-07: [Multi-agent systems need continuous eval pipelines](https://newruntime.com/signals/continuous-evals-for-multi-agent-systems/)
- 2026-07-06: [Agent memory is being evaluated as system behavior](https://newruntime.com/signals/agent-memory-evaluated-as-system-behavior/)
- 2026-07-03: [Harvey trained a legal agent with an applied compute loop](https://newruntime.com/signals/harvey-trains-legal-agent-with-applied-compute/)
- 2026-07-02: [Verifiability is an AI product feature](https://newruntime.com/signals/verifiability-is-an-ai-product-feature/)
- 2026-06-29: [Prompt debt turns natural language into legacy code](https://newruntime.com/signals/prompt-debt-becomes-legacy-code/)
- 2026-06-23: [AI compresses coding, not the whole software lifecycle](https://newruntime.com/signals/ai-compresses-coding-not-whole-sdlc/)

## Public Sources

- https://addyo.substack.com/p/own-the-outer-loop
- https://addyosmani.com/blog/new-sdlc-vibe-coding
- https://anthropic.com/research/AI-assistance-coding-skills
- https://arxiv.org/abs/2602.12670
- https://arxiv.org/abs/2603.29919
- https://arxiv.org/abs/2605.24050
- https://databricks.com/blog/benchmarking-coding-agents-databricks-multi-million-line-codebase
- https://dbreunig.com/2026/06/22/the-problem-is-prompt-debt.html
- https://developers.openai.com/api/docs/guides/latest-model
- https://github.com/benchflow-ai/skillsbench
- https://github.com/OpenDataBox/MemoryData
- https://github.com/shadcn/improve