AI Memory vs. RAG vs. Vector DB: When You Need Which

"Just use a vector database." "Isn't that just RAG?" "Doesn't the model already have memory?" These three keep getting collapsed into one, and the confusion leads to real architecture mistakes. Here's the clean version.

Vector database: storage

A vector DB stores embeddings and returns nearest neighbours. It's infrastructure — fast similarity search over vectors. It has no opinion about what those vectors mean, whether they're still true, or who they belong to. Powerful, but it's a component, not a memory system, the same way a filesystem isn't a document editor.

RAG: retrieval over a corpus

Retrieval-augmented generation pulls relevant chunks from a document set and stuffs them into the prompt. It's the right tool when the knowledge lives in a stable corpus — docs, policies, a knowledge base. But RAG retrieves documents; it doesn't learn about a user. It has no notion of what happened last Tuesday, no provenance chain for a claim you made, and no sense that a fact changed.

RAG retrieves what's written down. Memory understands what happened.

Memory: durable, evolving knowledge about a subject

A memory layer is dynamic where RAG is static. It observes activity, distils it into structured, per-subject knowledge, tracks how facts change over time, carries provenance, and can be scoped and erased. Crucially, it manages a lifecycle — deciding what to keep, revising beliefs when facts change, and (in MemMesh) predicting from the accumulated history. A vector DB might sit underneath it; RAG might sit beside it; but memory is the layer that knows you.

A decision guide

• Need fast similarity search as a primitive? Vector DB.
• Need answers grounded in a fixed document corpus? RAG.
• Need the system to learn about a user, persist across sessions, track changing facts, and act on them? Memory.

They compose

This isn't a cage match. A production system often uses all three: a vector DB as storage, RAG for corpus-grounded facts, and a memory layer for the durable, personal, evolving context — plus, if you want foresight, prediction on top. The mistake isn't picking one; it's using RAG where you needed memory and wondering why the agent still feels like it just met you.