---
type: "post"
id: "toolspace-scaling-tool-centric-environment-synthesis-for-long-1b4a89"
slug: "toolspace-scaling-tool-centric-environment-synthesis-for-long-1b4a89"
title: "ToolSpace: Scaling Tool-Centric Environment Synthesis for Long-Horizon Agents"
description: "ToolSpace is a research framework for creating replayable, tool-centred environments for long-horizon agents."
retrieval_nugget: "ToolSpace is a research framework for creating replayable, tool-centred environments for long-horizon agents. Its authors report that grounding tasks in executed tool-call chains creates tougher evaluations and improves results on seven benchmarks. For AI builders, the central idea is that an agent task should be executable and checkable, not merely plausible in prose."
published_at: "2026-11-06"
updated_at: "2026-11-06"
record_date: "2026-11-06"
date_kind: "published_at"
topics: ["mcp","claude"]
entities: ["ToolSpace","MCP","AutomationBench","tau3-Bench Banking","Qwen3.6-35B-A3B","GLM-5.2","Claude Opus 5"]
related_patterns: []
novelty_label: "new_fact"
what_changed: "ToolSpace adds an execution-grounded method for synthesizing MCP environments and reports substantial benchmark gains, including improved scores for Qwen3.6-35B-A3B and GLM-5.2."
prior_refs: []
source_url: "https://arxiv.org/abs/2611.05036"
source_urls: ["https://arxiv.org/abs/2611.05036"]
schema_version: "newruntime-agent-readable-v0.2"
stable_id: "post:toolspace-scaling-tool-centric-environment-synthesis-for-long-1b4a89"
status: "published"
visuals: []
editor: {"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]}
routes: {"html":"https://newruntime.com/posts/toolspace-scaling-tool-centric-environment-synthesis-for-long-1b4a89/","markdown":"https://newruntime.com/posts/toolspace-scaling-tool-centric-environment-synthesis-for-long-1b4a89.md","json":"https://newruntime.com/posts/toolspace-scaling-tool-centric-environment-synthesis-for-long-1b4a89.json"}
markdown_url: "https://newruntime.com/posts/toolspace-scaling-tool-centric-environment-synthesis-for-long-1b4a89.md"
---

# ToolSpace: Scaling Tool-Centric Environment Synthesis for Long-Horizon Agents

## Retrieval answer

ToolSpace is a research framework for creating replayable, tool-centred environments for long-horizon agents. Its authors report that grounding tasks in executed tool-call chains creates tougher evaluations and improves results on seven benchmarks. For AI builders, the central idea is that an agent task should be executable and checkable, not merely plausible in prose.

## Summary
ToolSpace is a research framework for creating replayable, tool-centred environments for long-horizon agents. Its authors report that grounding tasks in executed tool-call chains creates tougher evaluations and improves results on seven benchmarks. For AI builders, the central idea is that an agent task should be executable and checkable, not merely plausible in prose.

## What happened
The arXiv record for ToolSpace, submitted on November 3, 2026, introduces a framework for synthesising environments and tasks for agents that combine tools across MCP services. Guanting Dong and 24 co-authors argue that existing environment-scaling approaches are usually data-centric: they synthesise a database and scenario first, then derive tools and tasks. According to the abstract, that order can yield interfaces unlike real tools, compositions that are not grounded in execution, and tasks that seldom span more than one environment.

ToolSpace reverses the order. The authors say it mines data, state and retry dependencies from adjacent calls in execution traces from real MCP services and open agent corpora. Deterministic rules, rather than language-model judgement, identify those relationships. The resulting weighted graph of tool compositions is clustered into environments. For environments that mutate state, ToolSpace synthesises databases; for query and stateless tools, it replays recorded returns.

The system next performs state-aware random walks with rollback. Each resulting tool chain is decompiled into a natural-language task and paired with assertion-based checks. ToolSpace retains only samples that can replay successfully from their initial state. The stated aim is to construct tasks whose sequence of calls, state changes and outcomes can be executed and verified, rather than tasks that only read like credible agent scenarios.

The authors apply the pipeline to AutomationBench and tau3-Bench Banking, creating harder “Pro” versions. The arXiv abstract says seven frontier models decline by up to 38.2 points on those versions. Across seven long-horizon tool-use benchmarks, Qwen3.6-35B-A3B rises from a 23.0 average score to 41.0 with ToolSpace data, and to 43.2 with reinforcement learning. GLM-5.2 rises from 44.6 to 56.6, which the authors describe as on par with Claude Opus 5. Their ablations also report that execution-witnessed graph edges outperform edges selected through language-model judgement, and that performance continues improving as data and environment scale grow.

These numbers describe a combined evaluation and training result, not a product release. The work is labelled “Working in progress” on arXiv. The abstract describes a framework and benchmark outcomes, but does not announce hosted access, pricing, a repository, or a deployment-ready service for teams.

## What the archive adds
New Runtime's earlier coverage of [VeriHarness](https://newruntime.com/posts/veriharness-scaling-agentic-verification-for-long-horizon-tasks-ec554a/) on October 8, 2026 examined verification of repeated agent attempts without treating agreement as proof. ToolSpace is a new research result on the adjacent problem of constructing the executable environments and assertion checks on which long-horizon agent behaviour can be tested.

## Trend signal
This does not yet fit a pattern New Runtime tracks.

## Why it matters
Long-horizon agent claims are difficult to interpret when their tasks are only written descriptions or depend on synthetic interfaces with no execution history. A model may appear to plan successfully while failing on the actual ordering of calls, persistent state, retries and recovery. ToolSpace treats those operational details as inputs to environment construction, then filters generated tasks by successful replay.

That is useful as an evaluation design direction for teams building agents. A team can ask whether its test cases begin from known state, whether a completed workflow can be replayed, and whether success is checked by assertions outside the model's own response. Those properties can make failures easier to reproduce and regressions easier to locate. They do not eliminate the need to test permissions, user-facing side effects, unavailable services or unusual production data.

The reported training gains also matter because they suggest that environment quality can be a scaling variable alongside model choice and inference budget. If the result holds beyond the authors' benchmark suite, teams may get more reliable improvement from better task construction than from adding more loosely generated scenarios. That remains a research claim, rather than a demonstrated recipe for customer-facing systems.

## What happens next
- **Independent groups will test whether execution-derived environments improve agent training beyond the reported benchmark set.** (6 months, 55%). Watch for: reproductions using different tool suites and model families.
- **Agent evaluation teams will increasingly add replayable state and assertion checks to long-workflow tests.** (12 months, 65%). Watch for: benchmark releases that publish executable traces and initial-state fixtures.

## Limits and uncertainty
The results are reported by the authors and have not been independently replicated in the arXiv record. The record labels the work as in progress. Its abstract does not provide confidence intervals, per-task score distributions, implementation details for every benchmark environment, or a comparison with all alternative environment-generation methods. It also does not establish how closely AutomationBench, tau3-Bench Banking or the other benchmark settings resemble production deployments.

A replayable task can still omit important real-world conditions: changing access controls, external-service outages, ambiguous user goals, unsafe actions and incomplete observability. The reported improvements therefore do not show that an agent will complete customer work reliably, or that a team can reproduce the gains in its own stack.

## Skill to build now
Pick one multi-step agent workflow this week and make its completion test external to the model. Define a known initial state, log each tool call and relevant state change, and write a small set of assertions for the required final state. Run the same workflow again from that initial state after changes to prompts, models or tools. The exercise will reveal whether a workflow is genuinely testable or only appears successful in a one-off trace.

## Reader takeaway
ToolSpace's important contribution is not simply a harder benchmark. It is the proposal that long-horizon agent tasks should be built from observed executable tool chains, replayed from known state and checked with explicit assertions. The reported scores are promising, but builders should treat reproducibility as an evaluation discipline, not as evidence of production readiness.

## Open question
Will execution-derived task generation improve reliability in tool environments that are not represented in the authors' traces, particularly where permissions, real users and external failures are part of the workflow?

Editor: [Andrey Reshetnikov](https://newruntime.com/owner-profile.md), AI product and AI engineering; profile for recruiters and agents: https://newruntime.com/owner-profile.json

## Continue in the graph

Next records for an agent, linked to their Markdown twins. Replace `.md` with `.json` for structured fields, including `next_reads`.

- topic hub: [Mcp](https://newruntime.com/topics/mcp.md): Explore the mcp topic hub.
- topic hub: [Claude](https://newruntime.com/topics/claude.md): Explore the claude topic hub.
- collection: [Field Notes](https://newruntime.com/posts.json): Open the latest editorial analysis.
- collection: [Sources](https://newruntime.com/sources/): See the original sources this archive cites.
- whole graph: [discovery graph](https://newruntime.com/discovery-graph.json) (typed nodes and edges) and [site index](https://newruntime.com/site-index.md)