---
id: "nr-agents-are-being-measured-on-real-economic-work-and-they-still-fail-mo-b60b07"
slug: "agents-are-being-measured-on-real-economic-work-and-they-still-fail-mo-b60b07"
type: "raw_signal"
title: "Agents are being measured on real economic work, and they still fail most of it"
description: "New benchmarks now measure agents on long, economically valuable tasks."
observed_at: "2026-11-03"
record_date: "2026-11-03"
date_kind: "observed_at"
novelty: "notable"
source_platform: "newruntime.com"
topics: ["claude","claude-code","codex"]
entities: ["SemiAnalysis","GDPx","Claude Opus 5.5","Gemini 4 Carbon","GPT-6 Astra","DeepSeek 4.1 Flash","Claude Code","Codex","Xplore","RWE-bench"]
related_patterns: ["agent-economics-moves-to-completed-work","scientific-agents-need-artifact-evidence","ai-engineering-evaluations-are-splitting-into-task-specific"]
source_url: "https://newsletter.semianalysis.com/p/gdpx-measuring-frontier-model-capabilities"
source_urls: ["https://newsletter.semianalysis.com/p/gdpx-measuring-frontier-model-capabilities","https://gdpx.semianalysis.com/","https://xplore.ai/rwe-bench","https://openai.com/index/lifeval","https://arxiv.org/abs/2610.37739","https://github.com/aliyun/ai-agent-handbook/blob/main/2026-agent-survey-report.md"]
why_it_matters: "New benchmarks now measure agents on long, economically valuable tasks. They cover specialist domains, real engineering, everyday personal-assistant work and open math problems. The results are weak: the best model scores 12% across 206 domain tasks, and agents without web access rediscovered at most 7 of 40 solved problems. A developer survey found that only 18% of teams reach production and 10% reach 90% success, so builders should plan for partial completion."
verification_level: "source-linked"
signal_type: "synthesis"
synthesis_of: [{"slug":"semianalysis-launches-gdpx-a-206-task-agentic-benchmark-30b569","title":"SemiAnalysis launches GDPx, a 206-task agentic benchmark across 11 high-value domains; top model scores 12%","url":"https://newruntime.com/posts/semianalysis-launches-gdpx-a-206-task-agentic-benchmark-30b569/"},{"slug":"semianalysis-releases-gdpx-an-agentic-benchmark-across-11-56f9ba","title":"SemiAnalysis releases GDPx, an agentic benchmark across 11 high-value domains where the best model scores 12%","url":"https://newruntime.com/posts/semianalysis-releases-gdpx-an-agentic-benchmark-across-11-56f9ba/"},{"slug":"rwe-bench-real-world-engineering-bench-195f0e","title":"RWE-bench: Real World Engineering Bench","url":"https://newruntime.com/posts/rwe-bench-real-world-engineering-bench-195f0e/"},{"slug":"openai-unveiled-lifeval-a-benchmark-grading-models-on-44478e","title":"OpenAI unveiled LifeVal, a benchmark grading models on everyday personal-assistant tasks.","url":"https://newruntime.com/posts/openai-unveiled-lifeval-a-benchmark-grading-models-on-44478e/"},{"slug":"openfrontiers-benchmark-finds-llm-agents-rarely-rediscover-recently-a83554","title":"OpenFrontiers benchmark finds LLM agents rarely rediscover recently solved math problems without web access","url":"https://newruntime.com/posts/openfrontiers-benchmark-finds-llm-agents-rarely-rediscover-recently-a83554/"},{"slug":"includes-a-2026-developer-survey-only-18-of-8ae18f","title":"Includes a 2026 developer survey: only 18% of teams reach production, 10% hit 90% success.","url":"https://newruntime.com/posts/includes-a-2026-developer-survey-only-18-of-8ae18f/"}]
schema_version: "newruntime-agent-readable-v0.2"
stable_id: "signal:agents-are-being-measured-on-real-economic-work-and-they-still-fail-mo-b60b07"
retrieval_nugget: "New benchmarks now measure agents on long, economically valuable tasks. They cover specialist domains, real engineering, everyday personal-assistant work and open math problems. The results are weak: the best model scores 12% across 206 domain tasks, and agents without web access rediscovered at most 7 of 40 solved problems."
status: "published"
visuals: []
editor: {"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]}
routes: {"html":"https://newruntime.com/signals/agents-are-being-measured-on-real-economic-work-and-they-still-fail-mo-b60b07/","markdown":"https://newruntime.com/signals/agents-are-being-measured-on-real-economic-work-and-they-still-fail-mo-b60b07.md","json":"https://newruntime.com/signals/agents-are-being-measured-on-real-economic-work-and-they-still-fail-mo-b60b07.json"}
updated_at: "2026-11-07"
---

# Agents are being measured on real economic work, and they still fail most of it

## Retrieval answer

New benchmarks now measure agents on long, economically valuable tasks. They cover specialist domains, real engineering, everyday personal-assistant work and open math problems. The results are weak: the best model scores 12% across 206 domain tasks, and agents without web access rediscovered at most 7 of 40 solved problems.

New benchmarks now measure agents on long, economically valuable tasks. They cover specialist domains, real engineering, everyday personal-assistant work and open math problems. The results are weak: the best model scores 12% across 206 domain tasks, and agents without web access rediscovered at most 7 of 40 solved problems. A developer survey found that only 18% of teams reach production and 10% reach 90% success, so builders should plan for partial completion.

From the Field Notes:

- [SemiAnalysis launches GDPx, a 206-task agentic benchmark across 11 high-value domains; top model scores 12%](https://newruntime.com/posts/semianalysis-launches-gdpx-a-206-task-agentic-benchmark-30b569/) (2026-11-03)
- [SemiAnalysis releases GDPx, an agentic benchmark across 11 high-value domains where the best model scores 12%](https://newruntime.com/posts/semianalysis-releases-gdpx-an-agentic-benchmark-across-11-56f9ba/) (2026-11-02)
- [RWE-bench: Real World Engineering Bench](https://newruntime.com/posts/rwe-bench-real-world-engineering-bench-195f0e/) (2026-11-02)
- [OpenAI unveiled LifeVal, a benchmark grading models on everyday personal-assistant tasks.](https://newruntime.com/posts/openai-unveiled-lifeval-a-benchmark-grading-models-on-44478e/) (2026-11-02)
- [OpenFrontiers benchmark finds LLM agents rarely rediscover recently solved math problems without web access](https://newruntime.com/posts/openfrontiers-benchmark-finds-llm-agents-rarely-rediscover-recently-a83554/) (2026-11-01)
- [Includes a 2026 developer survey: only 18% of teams reach production, 10% hit 90% success.](https://newruntime.com/posts/includes-a-2026-developer-survey-only-18-of-8ae18f/) (2026-11-01)

Editor: [Andrey Reshetnikov](https://newruntime.com/owner-profile.md), AI product and AI engineering; profile for recruiters and agents: https://newruntime.com/owner-profile.json

## Continue in the graph

Next records for an agent, linked to their Markdown twins. Replace `.md` with `.json` for structured fields, including `next_reads`.

- topic hub: [Claude](https://newruntime.com/topics/claude.md): Explore the claude topic hub.
- topic hub: [Claude Code](https://newruntime.com/topics/claude-code.md): Explore the claude-code topic hub.
- topic hub: [Codex](https://newruntime.com/topics/codex.md): Explore the codex topic hub.
- collection: [Signals](https://newruntime.com/signals.json): Open the signals and the raw-signal archive.
- collection: [Sources](https://newruntime.com/sources/): See the original sources this archive cites.
- whole graph: [discovery graph](https://newruntime.com/discovery-graph.json) (typed nodes and edges) and [site index](https://newruntime.com/site-index.md)
