A Vector Store Is Not An Agent Memory System

Contextual AI separates working, procedural, semantic, and behavioral memory, with evaluation and provenance gates protecting every durable write.

Retrieval answer

Contextual AI separates working, procedural, semantic, and behavioral memory, with evaluation and provenance gates protecting every durable write. External memory is often more practical than training facts and behavior into model weights. It can be inspected, updated, validated, and carried across model upgrades.

New Runtime synthesiseditorial-diagram
Hand-drawn four-layer memory architecture separating current task context, tool-use playbooks, stable domain facts, and production behavior patterns, with provenance and evaluation gates controlling durable updates.
Different memories require different storage, retrieval, update, and validation rules; similarity search is only one component.New Runtime synthesis from Contextual AIOriginal source ↗
  1. WorkingCurrent requests, evidence, tool results, and intermediate state share a finite context budget.
  2. ProceduralValidated playbooks capture how this agent should search and use its tools.
  3. SemanticStable domain facts and authoritative-source rules persist outside individual sessions.
  4. BehavioralRepeated production feedback changes prompts and response behavior after evaluation.

External memory is often more practical than training facts and behavior into model weights. It can be inspected, updated, validated, and carried across model upgrades. Contextual AI argues that this flexibility requires a real write and retrieval policy rather than a vector database filled with old interactions.

A vector index can retrieve something similar. It does not decide what deserves to be remembered, when information has expired, how a noisy trace becomes a durable lesson, whether a proposed update is safe, or how evidence survives compaction. Those decisions form the memory system.

Contextual separates four layers. Working memory holds the current request, recent turns, tool results, intermediate state, and source evidence under a finite context budget. Procedural memory stores validated lessons about how to search a particular corpus. Semantic memory holds stable domain facts and authoritative-source rules. Behavioral memory captures recurring patterns in production feedback and updates how the agent responds.

The separation prevents a correction from automatically becoming the wrong kind of state. Not every interaction belongs in an embedding, every lesson in a prompt, or every user preference in long-term behavior. Tool traces can improve a procedural playbook, while domain definitions need provenance and a different lifecycle.

Writes are the higher-risk operation. Retrieved memories become stale, summaries discard decisive evidence, prompt memory competes with task context, and optimization loops overfit narrow evaluations. Contextual therefore places evaluation gates before updates, relevance checks on retrieval, provenance on semantic facts, and citation preservation inside compaction.

For New Runtime, memory should be modeled as typed records with separate owners and promotion rules. The immediate audit is straightforward: identify which state is session-only, which operational lessons can be proposed from traces, which facts need source lineage, and which production patterns may change behavior only after an evaluation proves the change helps.

Recommendation

Contextual AI separates working, procedural, semantic, and behavioral memory, with evaluation and provenance gates protecting every durable write.

Discovery graph / next reads

Continue through New Runtime

Open the graph
  1. 01topicAgents - New RuntimeExplore the agents topic hub.
  2. 02topicContext engineering - New RuntimeExplore the context engineering topic hub.
  3. 03related materialPatientAgentBench Tests Health Agents As WorkflowsShares agents and evals.
  4. 04related materialRamp Separates Agent Reasoning From Risk DecisionsShares agents and evals.
  5. 05related materialFirecrawl MCP Turns Web Search into a Bounded Agent CapabilityShares agents and context engineering.

These links are also published in this page’s JSON twin and as typed edges in DiscoveryGraph v1.

Who read this page?Machine requests, hidden until opened

Loading the privacy-safe route aggregate…

Open the JSON contract