What problems agent memory solves
- Continuity: users shouldn’t repeat themselves in every session.
- Currency: when a fact changes, the agent should use the new value.
- Grounding: answers about the past should point to evidence.
- Boundaries: what one user told the agent shouldn’t leak to another.
Agent memory vs a context window
A context window is what the model reads on one call. Memory decides what goes into it. Even very large windows don’t remember between sessions, cost more as they fill, and don’t resolve which of two conflicting statements is current. Memory is selection; the window is capacity.
The vocabulary: working, episodic, semantic, procedural
| Type | What it holds | Example |
|---|---|---|
| Working | The current task state | Steps done in this run |
| Episodic | Specific past events, with time | “On 12 Sep the customer asked to delay renewal.” |
| Semantic | Facts and beliefs distilled from events | “Renewal is on 1 November.” |
| Procedural | How to do things | A team’s deploy checklist |
Agent memory vs RAG
RAG retrieves passages similar to a query from a corpus. Memory adds state: it tracks entities and facts over time, supersedes outdated values, and filters by viewer. Many systems use both — RAG for documents, memory for what happened.
Common architectures
- Transcript replay: paste history into the prompt. Simple, expensive, degrades with length.
- Summaries: compress history. Cheap, loses detail and dates.
- Vector memory: embed messages, retrieve by similarity. Good recall, weak on updates.
- Structured or temporal memory: extract facts with time and source, reconcile changes. More engineering, better for changing facts.
What to require from a memory layer
Time
Store when things happened, and answer as of a date.
Updates
Prefer newer facts, keep history.
Provenance
Return the source for each memory.
Scope
Filter by who is asking.
Deletion
Forget records and everything derived from them.
Frequently asked questions
Is fine-tuning a form of memory?
It changes a model’s general behaviour, but it is slow to update, hard to delete from and not per-user. It is a poor fit for facts that change.
Do I need memory for a single-session chatbot?
Usually not. Memory matters once sessions, users or systems multiply.