AWS and UC Berkeley's Hydro project released Infinity on October 23, 2026, an early research Rust runtime that suspends an agent at every tool call and keeps only its conversation history while it waits. AWS headlined that 75,000 concurrent agents fit in the 8 GB of a Raspberry Pi 5, extrapolated from a single-thread server benchmark.
An independent rerun by AgentOps Guide matched the per-agent figure but showed capacity depends on history length. For builders sizing long-lived, wait-heavy agents, the question is whether either number transfers to real models, durable stores and compaction.
What we checked
- How do Infinity's memory and throughput curves change with real model calls, persistent storage, compaction, and concurrent production workloads? unresolved. No published benchmark covers any of these conditions. AWS's own test and the AgentOps Guide rerun both used a scripted model and in-memory stores, and the rerun ran with compaction disabled; AWS publishes no real-model or storage-backed figures. [1][3][2]
- Does the independent rerun reproduce AWS's per-agent memory rate? confirmed. The rerun measured 105.6 KB per agent at 20 turns against AWS's stated about 100 KB, and found memory scales roughly linearly with turns at about 5 KB each. [3][1]
- Would Infinity's built-in compaction bound per-agent memory in long-running deployments? partly confirmed. Docs say the local driver compacts at roughly three quarters of the model's context window, and AWS says automatic compaction is a local-mode feature. The effect on memory, latency and answer quality has not been measured, and AWS says compaction for large agent trees is unstudied. [4][1]
What we found
- AWS's figure comes from a benchmark of 80,000 agents on one EPYC thread with in-memory stores and a simulated model, at about 100 KB per idle agent; the Pi number is arithmetic on that rate. The benchmark source confirms a scripted in-process model and RSS sampled only when every agent is idle. [1][2]
- The AgentOps Guide rerun reproduced the baseline (10,000 agents, 20 turns, 105.6 KB each) and then varied history length from 5 to 160 turns. Memory rose about 5 KB per tool-calling turn, so the 8 GB capacity fell to roughly 10,100 agents at 160 turns, and single-thread throughput fell from about 18,200 to 2,500 completions per second. [3]
- The rerun ran with compaction off. Infinity's docs say the local driver compacts at roughly three quarters of the context window, and AWS describes automatic compaction as a local-mode addition, so neither published curve shows the bounded-history case. AWS itself lists compaction for long-running agent trees as unstudied. [3][4][1]
- In the serverless reference deployment, histories live in Aurora DSQL, so idle agents hold no process memory at all and the resident-memory curve stops being the constraint. Each resume reloads history and pays for queue, function and model calls, which no published benchmark has measured. [1][5]
What it means
Size Infinity by turns and tool-output bytes per agent, not by the headline agent count. The rerun's roughly 5 KB per turn is a floor from small scripted turns; real tool outputs, diffs and logs will push it higher until compaction kicks in, and compaction's threshold is set by the model's context window, which allows far more history than the 20-turn benchmark.
Before committing, rerun agent_scale with your own turn counts and output sizes, then repeat with a real context window and a durable store. On the serverless path, measure per-resume latency and storage cost against history size rather than resident memory.
Competing readings
- History length is the real capacity driver; local-mode deployments with realistic tool outputs will land well below 75,000 agents per 8 GB (likely, about 65%). What would tell it apart: In the rerun, memory rose about 5 KB per agent per turn and throughput fell as histories grew [3]; AWS's rate was measured at one short scripted history length, sampled only when agents were idle [1][2].
- For serverless step mode the memory curve is mostly moot; storage reads and model latency per resume become the real limits (unlikely, about 25%). What would tell it apart: In the reference deployment, histories are stored in Aurora DSQL, not in process memory [1]; Resuming an agent still has costs [5].
- Compaction keeps histories bounded, so the headline roughly holds for long-running local agents (very unlikely, about 10%). What would tell it apart: Compaction is built into local mode but triggers only near three quarters of the context window, far above the benchmark history [4][1]; Compaction was off in the rerun, and AWS has not studied it for large trees [3][1].
Base rate
The New Runtime archive holds only 4 earlier research stories about companies, organisations, products, projects or standards from the last 12 months that are at least 3 months old: too few (minimum 5) for a base rate. The archive has only 4 earlier comparable research stories, 2 of which were followed by a release. That is too few cases for a base rate, so this conclusion rests on the sources alone.
What would change this view
- AWS or an independent team publishes agent_scale results with compaction on at a real context window, showing per-agent memory leveling off near 100 KB (signal): Compaction bounds memory well enough that the headline is a fair planning figure for long-running agents.
- A published step-mode benchmark on Aurora DSQL shows per-resume latency staying flat as history grows (signal): History length matters less on the serverless path than the local-mode curve suggests.
- A new Infinity release changes how histories are stored, for example by compressing or offloading idle histories in local mode (open-source signal): The 5 KB per turn slope no longer describes the runtime.
Still open
- How do memory and throughput behave with compaction enabled at a realistic context window?
- What is per-resume latency and cost in step mode with Aurora DSQL as history grows?
- How does multi-threaded local mode scale under concurrent real model calls?
- How does compaction quality hold up for large, long-running agent trees?