---
type: "post"
id: "arena-ai-replaced-bash-recovery-with-bash-reliability-0fc075"
slug: "arena-ai-replaced-bash-recovery-with-bash-reliability-0fc075"
title: "Arena.ai replaced Bash Recovery with Bash Reliability , scoring first-try shell command accuracy."
description: "**Thesis:** Agent evaluation becomes more interpretable when a metric states which errors it attributes to the agent and which it excludes."
retrieval_nugget: "**Thesis:** Agent evaluation becomes more interpretable when a metric states which errors it attributes to the agent and which it excludes."
published_at: "2026-10-14"
updated_at: "2026-10-14"
record_date: "2026-10-14"
date_kind: "published_at"
topics: []
source_urls: ["https://arena.ai/resources/agent-arena-signals"]
schema_version: "newruntime-agent-readable-v0.2"
stable_id: "post:arena-ai-replaced-bash-recovery-with-bash-reliability-0fc075"
status: "published"
visuals: []
editorial_provenance: {"schema_version":"newruntime-editorial-copy-v1","content_status":"source_grounded_final","final_copy_sha256":"sha256:5f324d0b791d172517f9cce3a8262bccc920897837e0440254e7b62803f1567f","reviewed_at":"2026-10-14T05:02:39.026Z","source_evidence_count":1,"verified_claim_count":3,"site_analysis_schema_version":"newruntime-site-analysis-v1","site_object_kind":"field_note","observed_fact_count":3,"implication_count":1,"watch_condition_count":1,"related_record_count":0}
analysis: {"schema_version":"newruntime-site-analysis-v1","object_kind":"field_note","thesis":"Agent evaluation becomes more interpretable when a metric states which errors it attributes to the agent and which it excludes.","observed_facts":[{"text":"Arena.ai states that Bash Recovery is now called Bash Reliability.","source_urls":["https://arena.ai/resources/agent-arena-signals"]},{"text":"The source classifies bash calls as success, failure, or exempt; agent-caused errors count as failures and external errors such as network outages are exempt.","source_urls":["https://arena.ai/resources/agent-arena-signals"]},{"text":"The score is based on failures per non-exempt call; correctly run tests that find issues and searches that return no results count as successes.","source_urls":["https://arena.ai/resources/agent-arena-signals"]}],"mechanism":"The signal attributes failures to the agent when it causes an error, excludes errors outside its control, and computes failures per non-exempt bash call.","why_now":"Arena.ai identifies these definitions as the ground truth for turning live interactions into model-level Agent Arena signals, making the scoring rule relevant to interpreting its leaderboard.","implications":["Readers can distinguish model-attributed command failures from environmental failures when interpreting the signal."],"evidence_boundary":"The supplied public page defines the metric but provides no model scores, comparative results, or evidence that the change alters rankings.","watch_conditions":["Watch for Arena.ai to publish model-level Bash Reliability results or explain how the signal affects leaderboard comparisons."],"related_records":[],"new_branch_reason":"No prior New Runtime record was supplied; this distinct record captures the documented Bash signal rename and its scoring rules."}
editor: {"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]}
routes: {"html":"https://newruntime.com/posts/arena-ai-replaced-bash-recovery-with-bash-reliability-0fc075/","markdown":"https://newruntime.com/posts/arena-ai-replaced-bash-recovery-with-bash-reliability-0fc075.md","json":"https://newruntime.com/posts/arena-ai-replaced-bash-recovery-with-bash-reliability-0fc075.json"}
---

# Arena.ai replaced Bash Recovery with Bash Reliability , scoring first-try shell command accuracy.

## Retrieval answer

**Thesis:** Agent evaluation becomes more interpretable when a metric states which errors it attributes to the agent and which it excludes.

**Thesis:** Agent evaluation becomes more interpretable when a metric states which errors it attributes to the agent and which it excludes.

**Observed facts:** Arena.ai states that Bash Recovery is now called Bash Reliability. The source classifies bash calls as success, failure, or exempt; agent-caused errors count as failures and external errors such as network outages are exempt. The score is based on failures per non-exempt call; correctly run tests that find issues and searches that return no results count as successes.

**Mechanism:** The signal attributes failures to the agent when it causes an error, excludes errors outside its control, and computes failures per non-exempt bash call. **Why now:** Arena.ai identifies these definitions as the ground truth for turning live interactions into model-level Agent Arena signals, making the scoring rule relevant to interpreting its leaderboard.

**First implication:** Readers can distinguish model-attributed command failures from environmental failures when interpreting the signal. **Evidence boundary:** The supplied public page defines the metric but provides no model scores, comparative results, or evidence that the change alters rankings. **First watch condition:** Watch for Arena.ai to publish model-level Bash Reliability results or explain how the signal affects leaderboard comparisons.

This record starts a distinct analytical branch because no prior New Runtime record was supplied; it captures the documented Bash signal rename and scoring rules.

Editor: [Andrey Reshetnikov](https://newruntime.com/owner-profile.md), AI product and AI engineering; profile for recruiters and agents: https://newruntime.com/owner-profile.json

## Continue in the graph

Next records for an agent, linked to their Markdown twins. Replace `.md` with `.json` for structured fields, including `next_reads`.

- collection: [Field Notes](https://newruntime.com/posts.json): Open the latest editorial analysis.
- collection: [Source Ledger](https://newruntime.com/source-ledger.json): Inspect the public source evidence graph.
- whole graph: [discovery graph](https://newruntime.com/discovery-graph.json) (typed nodes and edges) and [site index](https://newruntime.com/site-index.md)