---
type: "post"
id: "stanford-open-sources-memorilla-latent-semantic-memory-that-035052"
slug: "stanford-open-sources-memorilla-latent-semantic-memory-that-035052"
title: "Stanford open-sources Memorilla, latent semantic memory that packs collections into 16 tokens"
description: "Stanford’s MEMORILLA learns to read an entire document collection and turn each query into a fixed set of memory tokens for a frozen language model."
retrieval_nugget: "Stanford’s MEMORILLA learns to read an entire document collection and turn each query into a fixed set of memory tokens for a frozen language model. The team reports strong results on eight benchmarks, including higher FactKG accuracy than full-context prompting with far fewer document tokens—a promising, but still experimental, alternative to retrieval alone."
published_at: "2026-11-07"
updated_at: "2026-11-07"
record_date: "2026-11-07"
date_kind: "published_at"
topics: ["memory"]
entities: ["Stanford University","MEMORILLA","Qwen3-8B","Qwen3-4B","Jure Leskovec"]
related_patterns: ["company-memory-needs-write-loops"]
novelty_label: "new_fact"
what_changed: "Stanford introduces Memorilla, a learned latent-memory module that compresses document collections into 16 query-dependent tokens and reports over 4,000× compression."
prior_refs: []
source_url: "https://memory.stanford.edu/"
source_urls: ["https://memory.stanford.edu/"]
schema_version: "newruntime-agent-readable-v0.2"
stable_id: "post:stanford-open-sources-memorilla-latent-semantic-memory-that-035052"
status: "published"
visuals: []
editor: {"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]}
routes: {"html":"https://newruntime.com/posts/stanford-open-sources-memorilla-latent-semantic-memory-that-035052/","markdown":"https://newruntime.com/posts/stanford-open-sources-memorilla-latent-semantic-memory-that-035052.md","json":"https://newruntime.com/posts/stanford-open-sources-memorilla-latent-semantic-memory-that-035052.json"}
markdown_url: "https://newruntime.com/posts/stanford-open-sources-memorilla-latent-semantic-memory-that-035052.md"
---

# Stanford open-sources Memorilla, latent semantic memory that packs collections into 16 tokens

## Retrieval answer

Stanford’s MEMORILLA learns to read an entire document collection and turn each query into a fixed set of memory tokens for a frozen language model. The team reports strong results on eight benchmarks, including higher FactKG accuracy than full-context prompting with far fewer document tokens—a promising, but still experimental, alternative to retrieval alone.

## Summary
Stanford’s MEMORILLA learns to read an entire document collection and turn each query into a fixed set of memory tokens for a frozen language model. The team reports strong results on eight benchmarks, including higher FactKG accuracy than full-context prompting with far fewer document tokens—a promising, but still experimental, alternative to retrieval alone.

## What happened
Stanford researchers describe MEMORILLA as a learned memory module for language models. It separates storing information from recalling it: a frozen encoder turns each incoming document into an embedding, while learned memory slots read both the question and the full embedding store. Those slots produce query-dependent memory tokens that a frozen LLM receives alongside the question. In supervised experiments the budget is 16 tokens, configurable in principle; for an agent experiment it is eight. The researchers train the memory module in stages, from paraphrasing Wikipedia through question answering and summarization to benchmark training splits. Adding a document costs one encoder pass, and does not require re-encoding earlier documents.

The Stanford announcement says that, across eight benchmarks using Qwen3-8B as the answer model, MEMORILLA has the highest mean score, 49.68, and leads on five benchmarks. It reports an average of 394 times fewer document tokens than full context. On FactKG, MEMORILLA scores 84.3% accuracy versus 80.4% for full context, while using 108 times fewer document tokens. The page also reports 44.6% short-answer accuracy on PersonaMem-v2, compared with 30.9% for the strongest baseline, xRAG. Results vary by task: on PubMedQA, the reported MEMORILLA score is 59.0, below full context and the best baseline.

For latency, the researchers measured 512 documents with Qwen3-8B at batch size one on a single H100-80GB: MEMORILLA took 0.74 seconds end to end, compared with 19.9 seconds for full context. The announcement says full context uses YaRN; it gives no RAG latency comparison. Its token and latency figures exclude the question and shared instructions. The reported 4,000-times-plus compression figure is for NarrativeQA relative to full context, not a general guarantee.

The team presents MEMORILLA as complementary to retrieval-augmented generation (RAG). RAG supplies selected source text, which can help when an answer requires an exact name. MEMORILLA reads the whole collection through its memory slots and may connect weakly related facts, but its compact representation can lose fine-grained details. Code, training recipes, and Hugging Face checkpoints are linked from the project page. The announcement describes the research and public artifacts; it does not announce a hosted service or commercial pricing.

## What the archive adds
New Runtime’s earlier coverage described MemU’s hierarchical memory, Zilliz’s semantic-highlighting approach to reducing RAG context, and Mastra’s observational memory. Those are adjacent approaches, not prior coverage of MEMORILLA: this is a new event and adds a learned, query-dependent fixed-token memory method with benchmark comparisons against retrieval and full context.

## Trend signal
MEMORILLA supports the view that agent memory needs a governed write and correction loop, rather than simply a larger searchable document pile. The project’s learned module updates an embedding store and compresses what a query needs, but the announcement does not establish production governance or correction behavior. See [shared-context infrastructure](https://newruntime.com/shifts/shared-context-infrastructure/).

## Why it matters
The useful result is not that 16 tokens can replace a large context window in every application. It is that a fixed-budget memory can sometimes preserve relationships across a large collection better than either full-context input or top-k retrieval, while lowering measured latency in one setup. For teams building assistants with persistent user history or agents that accumulate experience, this offers another design point: encode once, train a recall mechanism, and keep the model-facing representation bounded.

The trade-off is architectural. A compressed memory is not a faithful record: when users need exact names, quotations, or auditable facts, retrieved source text may be safer. Teams should treat learned memory as a candidate for synthesis and personalization, with retrieval or source inspection retained for verification. The results make this worth evaluating on a team’s own task and data, not deploying as a universal memory layer without testing.

## Limits and uncertainty
The reported results are experiments, not independent evaluations or evidence of production performance. The announcement says all experiments pair a Qwen encoder with a Qwen decoder on text; transfer to other model families and modalities is untested. MEMORILLA writes memory into input embeddings, so it requires open model weights and a memory module trained for each LLM. It can lose fine details such as exact names. The agent result comes from one game suite with episodes capped at 50 turns; long-running deployment, cross-environment learning, and performance on code or broader scientific work remain unshown. Latency also depends on the serving stack, and the page does not report a timed RAG comparison.

## Reader takeaway
MEMORILLA makes learned, fixed-budget memory a credible research alternative to “retrieve a few passages” or “put everything in context.” Its strongest case is connecting scattered information; its key risk is losing details that must be exact. Builders should compare it with retrieval on their own tasks and preserve a way to verify answers against source documents.

Editor: [Andrey Reshetnikov](https://newruntime.com/owner-profile.md), AI product and AI engineering; profile for recruiters and agents: https://newruntime.com/owner-profile.json

## Continue in the graph

Next records for an agent, linked to their Markdown twins. Replace `.md` with `.json` for structured fields, including `next_reads`.

- topic hub: [Memory](https://newruntime.com/topics/memory.md): Explore the memory topic hub.
- collection: [Field Notes](https://newruntime.com/posts.json): Open the latest editorial analysis.
- collection: [Sources](https://newruntime.com/sources/): See the original sources this archive cites.
- whole graph: [discovery graph](https://newruntime.com/discovery-graph.json) (typed nodes and edges) and [site index](https://newruntime.com/site-index.md)