How Does AI Memory Work? The Window, the Store, and the Retrieval

The assistant that forgets you by tomorrow

The same AI can feel razor-sharp at minute ten of a conversation and vaguely amnesiac the next day. It isn't getting worse, and you aren't imagining it: nothing about the model changed overnight. What changed is the context — and that's the clue to the whole subject. How does AI memory work? Every system that claims to remember is doing the same three-part trick: a window (what the model can see right now), a store (what survives after the conversation), and a retrieval step (what gets pulled back in when it matters). Models themselves are stateless — they don't hold onto anything between requests — so "AI memory" is always an external system wrapped around the model. Understand those three parts, and you understand every memory feature, every memory product, and every "but it forgot!" complaint on the internet.

Part 1: the window — what the model sees now

When you send a message to an AI, the model doesn't think in a vacuum. It reads a context window — a finite block of text that includes your latest message, the conversation so far (if any), and whatever else the system has injected. Everything the model "knows" in that moment is whatever is sitting in the window. Nothing more.

The window is the model's working memory, and it behaves exactly like a whiteboard:

• It has a fixed size (measured in tokens — words and word-parts).
• New input takes up space; when it's full, old content gets trimmed or compacted to make room.
• When the session ends, the whole thing is erased.

That's why a long conversation degrades: the earliest content — usually your most important instructions — is the first to be shed. It isn't a bug; it's the physics of a finite buffer. For the deeper mechanics of the window, including why "bigger" doesn't fix it, see context window vs context length and why AI has no memory.

Part 2: the store — what survives

If a model can't hold onto anything itself, memory has to live somewhere else. That's the store: a durable record that exists outside the window and survives the session. Stores come in many shapes, and the shape determines what the AI can do with it:

Store	What it holds	Typical use
Fact list	Short statements about the user ("works at X", "prefers Y")	ChatGPT-style memory features
File or document	Instructions, notes, project state (CLAUDE.md, HANDOFF.md)	Coding agents, persistent projects
Database or vector store	Records that can be searched semantically — past messages, documents, knowledge	Retrieval systems, long-term memory layers
Written summary	A compressed version of what happened	End-of-session handoffs (lossy by design)

The crucial detail: a store is only useful if something can read it later. A memory that nobody consults is a diary with a locked cover. That's where retrieval comes in.

Part 3: the retrieval — what comes back

A store can hold a million records and still be useless if the model can't find the right one at the right moment. Retrieval is the mechanism that decides what leaves the store and re-enters the window. The main strategies:

• Semantic search. The system finds records that mean something close to the current question, even when the words differ. This is what vector databases and embedding-based memory do — they compare meaning, not spelling.
• Recency. The newest records are assumed relevant and loaded automatically. Simple, and often right.
• Rules and priorities. Certain records are always loaded — a project's instruction file, a user's core preferences — because they matter every session.
• The agent's own judgment. The model decides what it needs, like a person flipping through their notes before a meeting.

Good memory systems combine these. Your most important facts load every time (rules); recent context follows you automatically (recency); and when you ask something specific, the system hunts through the whole history semantically (search). The art is in the mixing — too much retrieval floods the window with noise, too little starves the model of what it needs. That balance is the entire field of context engineering in one sentence.

The write path: how things get into the store

The part most people never see is the write. For memory to work, something has to decide what's worth saving, in a way that will still make sense later. Three approaches:

1. Explicit. You tell the AI what to remember. Highest precision, highest effort — you become the librarian.
2. Automatic capture. The system records what happened as it happens (episodic memory). Complete, but noisy — and if it records everything, retrieval has to fight through a lot of trivia.
3. Compression. At the end of a session, the AI writes a summary of what mattered. Compact, but every compression is a decision about importance — and a model that's already losing the thread is making that call. Repeated summaries drift. That's the summary drift problem in action, and it's why "just ask it to summarise" fails as a long-term strategy.

The strongest systems keep the source, not just a paraphrase. A record of what actually happened beats a retelling of it every time — which is why the best persistence layers store decisions and state directly rather than round-tripping them through summary.

Why models are stateless in the first place

There's a reason this architecture exists: models are functions. You give them input, they produce output, and they hold no state between calls. It's a feature for reliability — no hidden mutable brain to corrupt — but it means continuity has to be bolted on from the outside. Every "memory" you've ever seen in an AI product is that bolt: an external store plus a retrieval system, dressed up in settings and marketing.

Once you see it, the implications are clear:

• A model with a huge window still isn't remembering. It's holding more on the whiteboard. It gets erased the same way.
• A memory feature is only as good as its write and retrieval. Bad writes (summaries, inferred facts) and bad retrieval (everything or nothing) make even a perfect store useless.
• The architecture that actually persists keeps the source, writes deliberately, and retrieves by meaning — the difference between an assistant and a companion. That's the persistence thesis in full: why persistence beats a bigger context window.

The bottom line

How does AI memory work? Models are stateless; memory is external. A window shows the model what's happening now, a store keeps what matters after the session, and retrieval pulls the right records back at the right moment. Everything else — the features, the products, the forgetting — is a consequence of how well (or poorly) those three parts are built. Ask which of the three is failing, and you'll know exactly what to fix.

Read next: the types of agent memory, and RAG vs fine-tuning vs memory — when to use which.