Agentic Memory Architecture

Chat History is Not "Memory"

Agentic Memory Architecture

Appending {"role": "user", "content": "..."} to an array until you hit the token limit is not an architecture — it's a ticking time bomb. True Agentic Memory is split into Short-Term (operational) and Long-Term (persistent) subsystems.

Short-Term Memory (Working Context)

This is standard sliding-window context. But instead of blindly appending, Senior engineers use Token-Aware Summary Buffers. When the conversation history exceeds a threshold (e.g., 4000 tokens), an asynchronous background LLM distills the oldest 3000 tokens into a dense 200-token summary ("User and AI discussed Python deployment strategies..."). The new working context becomes: [Summary] + [Last 5 Messages].

Long-Term Memory (Persistent Storage)

What if a user mentions "I am allergic to peanuts" on day 1, and on day 400 asks for a recipe? Short-term memory has long forgotten this. Long-term memory continuously extracts semantic "Entities" and "Facts" from the user's conversation stream and writes them to a VectorDB or GraphDB.

Frameworks like Mem0 or Zep automatically monitor the stream, extracting facts (e.g. User_Allergy = Peanuts). When the user asks for a recipe later, the system queries the Long-Term memory vector store for relevant user facts and injects them into the system prompt dynamically.