Retrieval vs agent memory
View full-size diagram ↗What it is
Two ways an AI agent gets information beyond its prompt, and they're easy to confuse. Retrieval (the R in RAG) fetches relevant passages from an external knowledge base at query time, grounding an answer in facts the model was never trained on. Agent memory is state the agent writes and updates about a task or user. Retrieval answers 'what do I know?'; memory answers 'what happened before?'.
How it works
Retrieval embeds the question, returns the nearest chunks from a mostly read-only, shared corpus, and drops them into the window each call. Memory is read-write and per-agent: after an interaction, the system decides what to persist (a summary, a fact, an outcome), stores it, then recalls it later. MemGPT-style designs page memory in and out of the window like an OS swapping RAM.
A concrete example
A support agent handling 'why was I charged twice' uses retrieval to pull the current refund policy from the docs, knowledge it should never bake in because it keeps changing, and uses memory to recall that this same customer flagged the charge last week and a ticket is open. One turn, two mechanisms: retrieval supplies stable external facts, memory the evolving personal thread.
Why it matters
Reach for retrieval when the gap is 'the model doesn't know this fact', and for memory when it's 'the agent forgets across turns'. Pick wrong and you waste effort: piping raw chat logs into a vector store to fake memory returns stale fragments, while freezing changeable facts as memory means they rot when the source updates. Most long-running agents need both, wired separately.
What people get wrong: That memory is just retrieval run over past conversations, so a vector database of old messages is enough. Similarity search optimizes for sounding like the query; memory needs recency, salience, and consolidation, that is merging, updating, and forgetting. A nearest-neighbor lookup over history surfaces whatever reads as similar, not what keeps the agent consistent.
