---
id: "nr-agent-evaluation-is-shifting-to-harness-level-leaderboards-on-economic-8d6a2c"
slug: "agent-evaluation-is-shifting-to-harness-level-leaderboards-on-economic-8d6a2c"
type: "raw_signal"
title: "Agent evaluation is shifting to harness-level leaderboards on economically valuable work"
description: "Runta and Baseten's FrontierHarness Eval compares 22 harness configurations across terminal tasks, IBM and Kaggle's ScarfBench ranks agents on enterprise Java migration, SemiAnalysis's GDPx measures 206 long-horizon tasks across 11 high-value domains, and Joël Niklaus's leaderboard compares nine har"
observed_at: "2026-11-19"
record_date: "2026-11-19"
date_kind: "observed_at"
novelty: "notable"
source_platform: "newruntime.com"
topics: ["codex"]
entities: ["Runta","Baseten","FrontierHarness Eval","Harbor","Codex CLI","GLM-5.3","Kimi-K3","OpenCode","IBM","Kaggle"]
related_patterns: ["harness-architecture-outlives-model-choice","verification-bandwidth-is-the-scarce-resource","agent-economics-moves-to-completed-work"]
source_url: "https://runta.com/blog/introducing-frontierharness-eval-v1-1"
source_urls: ["https://runta.com/blog/introducing-frontierharness-eval-v1-1","https://research.ibm.com/blog/ibm-and-kaggle-launch-leaderboard-for-java-framework-migration","https://newsletter.semianalysis.com/p/gdpx-measuring-frontier-model-capabilities","https://huggingface.co/spaces/joelniklaus/harness-leaderboard"]
why_it_matters: "Runta and Baseten's FrontierHarness Eval compares 22 harness configurations across terminal tasks, IBM and Kaggle's ScarfBench ranks agents on enterprise Java migration, SemiAnalysis's GDPx measures 206 long-horizon tasks across 11 high-value domains, and Joël Niklaus's leaderboard compares nine harnesses with models fixed. The unit of evaluation is now the harness on real work, not the base model alone."
verification_level: "source-linked"
signal_type: "synthesis"
synthesis_of: [{"slug":"introducing-frontierharness-eval-v1-1-fd98d9","title":"Introducing FrontierHarness Eval v1.1","url":"https://newruntime.com/posts/introducing-frontierharness-eval-v1-1-fd98d9/"},{"slug":"ibm-and-kaggle-launch-java-framework-migration-leaderboard-ede4a9","title":"IBM and Kaggle launch Java framework migration leaderboard","url":"https://newruntime.com/posts/ibm-and-kaggle-launch-java-framework-migration-leaderboard-ede4a9/"},{"slug":"semianalysis-launches-gdpx-a-206-task-agentic-benchmark-30b569","title":"SemiAnalysis launches GDPx, a 206-task agentic benchmark across 11 high-value domains; top model scores 12%","url":"https://newruntime.com/posts/semianalysis-launches-gdpx-a-206-task-agentic-benchmark-30b569/"},{"slug":"joel-niklaus-releases-harness-leaderboard-comparing-nine-coding-f0efa5","title":"Joël Niklaus releases Harness Leaderboard comparing nine coding-agent harnesses on open models","url":"https://newruntime.com/posts/joel-niklaus-releases-harness-leaderboard-comparing-nine-coding-f0efa5/"}]
schema_version: "newruntime-agent-readable-v0.2"
stable_id: "signal:agent-evaluation-is-shifting-to-harness-level-leaderboards-on-economic-8d6a2c"
retrieval_nugget: "Runta and Baseten's FrontierHarness Eval compares 22 harness configurations across terminal tasks, IBM and Kaggle's ScarfBench ranks agents on enterprise Java migration, SemiAnalysis's GDPx measures 206 long-horizon tasks across 11 high-value domains, and Joël Niklaus's leaderboard compares nine harnesses with models fixed."
status: "published"
visuals: []
editor: {"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]}
routes: {"html":"https://newruntime.com/signals/agent-evaluation-is-shifting-to-harness-level-leaderboards-on-economic-8d6a2c/","markdown":"https://newruntime.com/signals/agent-evaluation-is-shifting-to-harness-level-leaderboards-on-economic-8d6a2c.md","json":"https://newruntime.com/signals/agent-evaluation-is-shifting-to-harness-level-leaderboards-on-economic-8d6a2c.json"}
markdown_url: "https://newruntime.com/signals/agent-evaluation-is-shifting-to-harness-level-leaderboards-on-economic-8d6a2c.md"
---

# Agent evaluation is shifting to harness-level leaderboards on economically valuable work

## Retrieval answer

Runta and Baseten's FrontierHarness Eval compares 22 harness configurations across terminal tasks, IBM and Kaggle's ScarfBench ranks agents on enterprise Java migration, SemiAnalysis's GDPx measures 206 long-horizon tasks across 11 high-value domains, and Joël Niklaus's leaderboard compares nine harnesses with models fixed.

Runta and Baseten's FrontierHarness Eval compares 22 harness configurations across terminal tasks, IBM and Kaggle's ScarfBench ranks agents on enterprise Java migration, SemiAnalysis's GDPx measures 206 long-horizon tasks across 11 high-value domains, and Joël Niklaus's leaderboard compares nine harnesses with models fixed. The unit of evaluation is now the harness on real work, not the base model alone.

From the Field Notes:

- [Introducing FrontierHarness Eval v1.1](https://newruntime.com/posts/introducing-frontierharness-eval-v1-1-fd98d9/) (2026-11-19)
- [IBM and Kaggle launch Java framework migration leaderboard](https://newruntime.com/posts/ibm-and-kaggle-launch-java-framework-migration-leaderboard-ede4a9/) (2026-10-31)
- [SemiAnalysis launches GDPx, a 206-task agentic benchmark across 11 high-value domains; top model scores 12%](https://newruntime.com/posts/semianalysis-launches-gdpx-a-206-task-agentic-benchmark-30b569/) (2026-11-03)
- [Joël Niklaus releases Harness Leaderboard comparing nine coding-agent harnesses on open models](https://newruntime.com/posts/joel-niklaus-releases-harness-leaderboard-comparing-nine-coding-f0efa5/) (2026-11-07)

Editor: [Andrey Reshetnikov](https://newruntime.com/owner-profile.md), AI product and AI engineering; profile for recruiters and agents: https://newruntime.com/owner-profile.json

## Continue in the graph

Next records for an agent, linked to their Markdown twins. Replace `.md` with `.json` for structured fields, including `next_reads`.

- topic hub: [Codex](https://newruntime.com/topics/codex.md): Explore the codex topic hub.
- collection: [Signals](https://newruntime.com/signals.json): Open the signals and the raw-signal archive.
- collection: [Sources](https://newruntime.com/sources/): See the original sources this archive cites.
- whole graph: [discovery graph](https://newruntime.com/discovery-graph.json) (typed nodes and edges) and [site index](https://newruntime.com/site-index.md)
