---
type: "post"
id: "frontiermemory-1-0-benchmarking-continual-learning-through-memory-d5a828"
slug: "frontiermemory-1-0-benchmarking-continual-learning-through-memory-d5a828"
title: "FrontierMemory 1.0: Benchmarking continual learning through memory."
description: "Letta Research has released FrontierMemory 1.0, a 15-task benchmark for judging agent memory by whether it improves later work rather than whether it can recall facts."
retrieval_nugget: "Letta Research has released FrontierMemory 1.0, a 15-task benchmark for judging agent memory by whether it improves later work rather than whether it can recall facts. For teams building agents, the result is a more demanding evaluation frame: memory must help an agent complete verified tasks, not merely retrieve a plausible note."
published_at: "2026-10-31"
updated_at: "2026-10-31"
record_date: "2026-10-31"
date_kind: "published_at"
topics: ["agent-memory","claude","claude-code"]
entities: ["Letta Research","FrontierMemory 1.0","Letta Dream","Letta Code","Claude Code","OpenAI Agents SDK","Hermes Agent","Prime Agent"]
related_patterns: ["company-memory-needs-write-loops","harness-architecture-outlives-model-choice","verification-bandwidth-is-the-scarce-resource"]
novelty_label: "new_fact"
what_changed: "FrontierMemory 1.0 reports three-run results with Opus 5.5 first at 73.3%; prior coverage describes the benchmark launch and earlier leaderboard results."
prior_refs: ["memory:1ac5cf0a-1cbb-8f6a-a487-fc16400ad8e9","memory:22027708-af3f-81ba-ac16-75268917a035"]
source_url: "https://frontiermemory.com/blog/v1"
source_urls: ["https://frontiermemory.com/blog/v1"]
schema_version: "newruntime-agent-readable-v0.2"
stable_id: "post:frontiermemory-1-0-benchmarking-continual-learning-through-memory-d5a828"
status: "published"
visuals: []
editor: {"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]}
routes: {"html":"https://newruntime.com/posts/frontiermemory-1-0-benchmarking-continual-learning-through-memory-d5a828/","markdown":"https://newruntime.com/posts/frontiermemory-1-0-benchmarking-continual-learning-through-memory-d5a828.md","json":"https://newruntime.com/posts/frontiermemory-1-0-benchmarking-continual-learning-through-memory-d5a828.json"}
---

# FrontierMemory 1.0: Benchmarking continual learning through memory.

## Retrieval answer

Letta Research has released FrontierMemory 1.0, a 15-task benchmark for judging agent memory by whether it improves later work rather than whether it can recall facts. For teams building agents, the result is a more demanding evaluation frame: memory must help an agent complete verified tasks, not merely retrieve a plausible note.

## Summary

Letta Research has released FrontierMemory 1.0, a 15-task benchmark for judging agent memory by whether it improves later work rather than whether it can recall facts. For teams building agents, the result is a more demanding evaluation frame: memory must help an agent complete verified tasks, not merely retrieve a plausible note.

## What happened

FrontierMemory 1.0 is a benchmark from Letta Research, released in October 2026, for evaluating continual learning through agent memory. Its central claim is that prevailing memory evaluations emphasize recall: an agent is asked about prior facts such as a person’s birthday or hobby. Letta argues that this is now an insufficient test. Models have improved, and coding harnesses can search files, so recall-oriented benchmarks such as LoCoMo and LongMemEval have largely been saturated without showing whether an agent can learn from experience and improve its later work.

The benchmark instead asks whether an agent can turn prior trajectories into useful memory and then use that memory on a downstream task. Letta describes this as the distinction between handing all relevant context to a new intern and allowing a senior employee to bring learned context to a new assignment. FrontierMemory adapts difficult, originally stateless tasks from DeepSWE, SkillsBench, AutomationBench, APEX-Agents, and EnterpriseOps-Gym so that successful completion requires or benefits from learned memory.

The 15 tasks cover three kinds of memory. Task memory concerns an underspecified contract or convention, such as omitting files listed in an `omit.txt` file when training a Word2Vec pipeline. User memory concerns recurring preferences inferred from feedback, such as producing a public API changelog in one line with the newest item first. Environment memory concerns non-obvious workspace or domain properties, such as finding grid limits in a particular environment file before calculating energy-grid curtailment. For every task, the team created a “gold memory” containing the additional information needed to establish that the task is solvable with memory.

Agents receive 665 trajectories from which to generate memory; only 219 are relevant to the evaluation tasks. The trajectories come from HarborIndex and DeepSWE, plus runs of harnesses including Codex, Claude Code, and OpenHands. Simulated users generate trajectories for user-memory tasks. At least two Letta researchers reviewed each task for sufficient source information, non-triviality of the memory-learning problem, and alignment between task requirements and the verifier. The review also found broken verifiers in 53% of the original sourced tasks retained for the benchmark: they could check for unrequested behavior, reject correct formatting, or demand actions unavailable through the task tools. Letta says it repaired those verifiers.

For model comparison, Letta Dream generated memories with seven models, while an agent running on Letta Code performed the benchmark tasks. The reported top model score is Opus 5.5 at 73.3% with a listed task cost of $3.42, followed by Kimi K3 at 71.1% ($4.16), GLM-5.3 at 68.9% ($4.08), and Gemini 4 Argon at 64.4% ($1.05). Scores are means across three independent runs. A harness comparison at maximum task effort reports, with the Opus 5.5 pair, Letta Dream at 73.3%, Hermes Agent at 64.4%, Claude Managed Agents at 55.6%, and Prime Agent at 53.3%. Letta also reports that instructions and imperative skill descriptions can make generated memory more usable than summaries alone.

## What the archive adds

This is a new version and a substantial new fact set on a known story, rather than an unrelated benchmark launch. The earlier Field Note, [Introducing FrontierMemory: a new way to evaluate agent memory through downstream task performance.](https://newruntime.com/posts/introducing-frontiermemory-a-new-way-to-evaluate-agent-43a03f/), established the benchmark’s downstream-performance premise. FrontierMemory 1.0 adds its 15-task construction, trajectory composition, verifier audit, rankings, costs, and harness comparison.

## Trend signal

The release supports [Company memory needs write and correction loops](https://newruntime.com/patterns/company-memory-needs-write-loops/). The benchmark treats memory as an artifact created from work histories and judged by a later task agent’s verified performance. Its verifier repairs also underline that a memory loop is only as useful as the task contract used to assess it; retrieval alone is not evidence of operational value.

## Why it matters

A team adding “memory” to an agent often has a weak success criterion: the system stores notes, retrieves them, and can repeat a past fact. FrontierMemory proposes a stricter one: can experience-derived state improve completion of a new task with a verifier? That change makes memory a systems problem spanning data selection, memory generation, instructions for using the memory, task harnesses, and evaluation.

The harness results also caution against treating the underlying model as the only design decision. On this benchmark, systems using matched memory and task models still differ substantially. Builders should therefore evaluate the path from trajectory to memory to action, including what the agent is told to do with the resulting artifact. The benchmark does not establish a universal winner, but it provides a concrete shape for tests that distinguish remembered information from usable learned behavior.

## What happens next

- **Agent-memory products will publish downstream-task evaluations rather than recall-only scores.** (6 months, 65%). Watch for: benchmarks that pair histories, generated memory, and executable task verifiers.
- **Harness design will become a first-class variable in memory comparisons.** (6 months, 60%). Watch for: matched-model studies reporting memory-generation and task-execution systems separately.

## Limits and uncertainty

FrontierMemory is a Letta Research benchmark and its reported scores, costs, task adaptations, and verifier repairs come from the benchmark’s own release. The material describes only 15 tasks and reports means over three independent runs, so it does not establish how rankings will transfer to a production agent’s users, tools, policies, or much longer history. The listed task-model costs are not a complete cost-of-ownership measure; the figure description says dreaming is excluded from its cost calculation. The source also does not provide an independent replication or show whether results persist as trajectory volume, task diversity, or adversarial memory inputs grow.

## Skill to build now

Create a small internal memory evaluation with one repeated workflow. Save a set of prior trajectories, identify the few facts or conventions a later task genuinely needs, and write a pass/fail verifier for that later task. Then compare a no-memory run with a memory-assisted run. Record not only whether the agent retrieved the right note, but whether the note changed the verified outcome. Keep user preferences, task contracts, and environment facts as separate categories so failures can be diagnosed rather than collapsed into a generic “memory” score.

## Reader takeaway

FrontierMemory 1.0 makes a useful demand of agent-memory systems: prove that learned state improves a later, verifiable task. For builders, the important unit is not a larger memory store or a stronger recall score, but an end-to-end loop from experience to a usable instruction or fact to completed work.

## Open question

Will the model and harness rankings remain stable when the history is longer, noisier, and drawn from real production workflows rather than benchmark trajectories?

Editor: [Andrey Reshetnikov](https://newruntime.com/owner-profile.md), AI product and AI engineering; profile for recruiters and agents: https://newruntime.com/owner-profile.json

## Continue in the graph

Next records for an agent, linked to their Markdown twins. Replace `.md` with `.json` for structured fields, including `next_reads`.

- topic hub: [Agent Memory](https://newruntime.com/topics/agent-memory.md): Explore the agent-memory topic hub.
- topic hub: [Claude](https://newruntime.com/topics/claude.md): Explore the claude topic hub.
- topic hub: [Claude Code](https://newruntime.com/topics/claude-code.md): Explore the claude-code topic hub.
- collection: [Field Notes](https://newruntime.com/posts.json): Open the latest editorial analysis.
- collection: [Source Ledger](https://newruntime.com/source-ledger.json): Inspect the public source evidence graph.
- whole graph: [discovery graph](https://newruntime.com/discovery-graph.json) (typed nodes and edges) and [site index](https://newruntime.com/site-index.md)