The repository describes 15 memory-dependent coding and knowledge-work tasks designed to test learned task contracts, user preferences, and environment rules.
A memory agent reads a fixed corpus of 674 trajectories, each with a PASS or FAIL verifier result, and writes a memory folder. A task agent then attempts the 15 tasks with that folder; the same task agent also runs without memory as a control. The reported score is the strict pass rate on task verifiers. Memory generation is limited to six hours and runs in a sandbox with access limited to the model API host.
The tooling requires Python 3.12+ plus Docker or Modal credentials. The task dataset is listed as letta-ai/frontier-memory version v1.0 on Harbor Hub, while the corpus is a gated Hugging Face dataset. The repository includes adapters and submission guidance for custom memory systems, and names Codex, Claude Code, Hermes, Letta Dream, OpenAI Agents SDK, Prime Agent, and Claude Managed Agents among harnesses covered by published comparisons.
The source does not establish that any listed harness will improve production outcomes, nor does it provide an independent validation of leaderboard results or a general measure of continual learning.