RAG vs Fine-Tuning vs Memory: When to Use Which

Ask it what you decided last week

You've built a RAG system — documents retrieved, answers grounded, citations working. Then you ask it: "What did we decide last week?" It stares back blankly. RAG retrieved your knowledge base; it has no idea what you decided, because that was never in the documents. That moment is the clearest introduction to RAG vs fine-tuning vs memory: three different tools for three different jobs, and only one of them remembers.

RAG (retrieval-augmented generation) brings external knowledge into the model's context on demand. Fine-tuning changes the model itself so it behaves a certain way. Memory persists state across sessions — who you are and what's happened. If you're building something with an AI that needs to know things, act a certain way, and remember you, the question isn't "which one" — it's whether you've built all three.

Here's the honest breakdown, and the decision rule that keeps you out of trouble.

RAG: knowledge on tap

RAG works by retrieving relevant documents or records at query time and injecting them into the model's context, so the model answers from material it can see rather than from memory alone.

Use RAG when the knowledge changes, lives outside the model, or needs to be auditable: - Your company's docs, a codebase, a manual, a changing knowledge base. - Facts that would go stale the moment they were baked into a model. - Anything where you need to show your work — "here's the source it came from."

Its strengths: no training required, update by swapping documents, grounded answers with citations, relatively cheap to change. Its limits: retrieval quality decides everything — bad search means bad answers, no matter how good the model is. And RAG gives the model facts; it doesn't give it a history with you. Ask a RAG system "what did we decide last week?" and it stares back blankly, because last week's decision wasn't in the documents.

That last point is where RAG gets overhyped. People bolt RAG onto a chatbot expecting continuity, and get a very well-informed amnesiac instead. Retrieval is not memory; it's lookup. The difference is the subject: RAG retrieves knowledge about the world, memory retrieves what happened with you. For the full anatomy of that gap, see how does AI memory work.

Fine-tuning: changing the model itself

Fine-tuning takes a base model and trains it further on your data, adjusting its weights so it behaves differently — a style, a format, a domain's voice.

Use fine-tuning when the pattern is fixed and repeated: - A consistent tone or format the model should produce every time ("always answer like a senior SRE", "always structure responses this way"). - A domain behaviour that doesn't change — medical triage phrasing, legal disclaimer style, a character's voice. - Latency or cost wins, if you're calling a tuned model at scale instead of stuffing prompts full of examples.

Its strengths: the behaviour is in the model, not in the prompt — no context budget spent on instructions, consistent output, faster and cheaper at scale. Its limits: expensive to build and update, static once trained, and — the part everyone forgets — fine-tuning is not a way to store facts about your users. A model tuned on your company's style guide still won't know what Daniel asked it on Tuesday. Weights are behaviour, not biography. If you fine-tune hoping for memory, you'll get a very polished amnesiac — the same failure as RAG, wearing a fancier price tag.

Memory: state that survives sessions

Memory is the layer that keeps what happened and who you are, stored outside the model and re-injected when relevant. It's the difference between an API that answers and an agent that continues.

Use memory when continuity matters: - A project that spans many sessions and can't afford re-explaining. - An assistant or companion that should know you. - An agent whose decisions need to survive from one run to the next.

Memory isn't knowledge retrieval (that's RAG) and it isn't behaviour (that's fine-tuning). It's the record of your actual working relationship — episodic, semantic, and procedural, in the terms of types of agent memory. Done well, it's also the layer that makes RAG feel intelligent: the system remembers what you've already asked, so it retrieves the next relevant thing instead of the same first page.

The comparison, side by side

	RAG	Fine-tuning	Memory
What it changes	What the model sees	What the model is	What the model keeps
Stores	External documents	Learned behaviour	Session history, facts, state
How it updates	Swap documents	Retrain (slow, costly)	Write a new record (instant)
Best for	Changing knowledge, citations	Fixed style and behaviour	Continuity across sessions
Biggest trap	Believing lookup = memory	Believing weights = memory	Bad write/retrieval design
When it's overkill	A single static FAQ	A prompt would do the job	One-off questions

The trap column is the real content of this page. All three technologies fail at memory when people reach for them as if they were memory. RAG retrieves, fine-tuning stylises — only a persistence layer remembers, and only because it was built to.

When to use which: the decision rule

Walk this ladder and you'll rarely go wrong:

1. Does the answer depend on documents that change? → RAG. Retrieve them at query time.
2. Does the output need a fixed, repeated style or behaviour? → Fine-tuning — but only after you've confirmed prompts and RAG can't do it, because they're cheaper to change.
3. Does the work span sessions and need to remember you, your decisions, your history? → Memory. This is the one almost everyone skips, and it's the one that makes the other two feel alive.
4. Do you need two? → Probably yes. The canonical stack is RAG for knowledge plus memory for continuity, with fine-tuning reserved for behaviour that truly needs to be in the weights.

If you're at step 3 and wondering what it looks like in practice, the pattern is the same one behind long-term memory for AI agents and the persistence thesis that a durable store beats a bigger context window.

The bottom line

RAG vs fine-tuning vs memory: RAG gives the model knowledge, fine-tuning gives it behaviour, and memory gives it a relationship with you. They're complements, not competitors — the mistake is using one where another belongs, and the most expensive version of that mistake is reaching for RAG or fine-tuning when what you actually need is to remember. Knowledge, style, and continuity are three different jobs. Hire for each.

Read next: how does AI memory work, and the types of agent memory.