Most AI agents forget everything between conversations. The architecture that fixes this — persistent, structured, retrievable memory — separates production systems from prototypes. Here is how the leading frameworks actually work, where they break, and what to build next.

The Memory Problem in Production

Large language models process tokens through a context window and then discard them. Every new conversation starts from zero. For a chatbot answering FAQs, that limitation is tolerable. For an agent managing customer relationships, triaging security alerts, or orchestrating multi-step workflows, stateless interaction is a design flaw.

The distinction between chat history and agent memory is well established. Chat history replays past messages. Agent memory stores, indexes, retrieves, and reasons over accumulated knowledge across sessions, users, and tasks. The challenge is not in storing information — any database can do that. The challenge is in retrieving the right memory at the right time, keeping it current, and preventing it from poisoning downstream decisions.

Memory without retrieval strategy is just a log file. The architecture matters because retrieval determines everything the agent can act on.

Production agent memory has to handle five distinct problems: context window overflow, cross-session persistence, temporal relevance decay, multi-user isolation, and adversarial injection. Each problem demands a different architectural pattern, and no single framework solves all of them simultaneously.

Five Memory Types, Five Architectural Patterns

Agent memory is not a monolith. The field has converged on five recognizable types, each with distinct storage, retrieval, and decay characteristics.

Memory Type What It Stores Retrieval Signal Decay Behavior Production Pattern
Short-term Current conversation turns, task context Recency + relevance Cleared per session Sliding window with summarization
Long-term User preferences, accumulated knowledge Semantic similarity Manual or policy-based expiry Vector store + metadata filter
Episodic Event sequences, outcomes, decisions made Temporal proximity + task match Summarized after threshold Time-indexed event log
Semantic Generalized facts, entity relationships Entity resolution + graph traversal Corrected, not decayed Knowledge graph with confidence scores
Procedural Learned workflows, tool-use sequences Task-type classification Replaced when superseded Template library + refinement loop

Most production systems combine two or three of these types. A customer service agent uses short-term memory for the current conversation, long-term memory for user profile data, and episodic memory for past interaction outcomes. A security triage agent relies on semantic memory for threat intelligence relationships and procedural memory for escalation workflows.

How the Leading Frameworks Actually Work

MemGPT / Letta: The Operating System Metaphor

MemGPT, presented at ICLR 2024 with 806 citations, introduced the idea that LLM memory management should mirror virtual memory in operating systems. The model manages its own context window through explicit memory functions — mem_insert, mem_search, mem_replace — rather than relying on a separate retrieval pipeline. The agent decides what to keep in working memory and what to page out to archival storage.

The core insight of MemGPT is that the model itself should control memory management, not an external orchestrator. The agent pages information in and out of its context window the way an OS pages between RAM and disk.

In practice, this means Letta (the commercial successor) stores a full conversation log plus extracted facts in a PostgreSQL-backed store, and the agent calls memory operations as tool invocations. Write latency is a known bottleneck — Letta documentation reports 4-6 second write times for complex memories, which limits throughput in high-volume scenarios. The open-source version is self-hosted; the managed Letta Cloud is in limited availability as of mid-2026.

Zep / Graphiti: Bi-Temporal Knowledge Graphs

Zep takes a fundamentally different approach. Rather than treating memory as document retrieval, Zep's Graphiti engine stores memories as nodes and edges in a knowledge graph, where every edge carries two timestamps: when the fact became true, and when the system learned it. This bi-temporal model enables queries like "what did we know about this user on March 15?" — a retrieval pattern that vector similarity alone cannot express.

Graphiti achieved a 94.8 percent score on the DMR (Dialog Memory Resolution) benchmark, outpacing MemGPT's 93.4 percent. Retrieval latency sits under 200 milliseconds for standard entity lookups. The tradeoff is graph complexity — managing schema evolution, handling contradictory facts, and resolving entity merging are all unsolved problems in open graphs at production scale.

Mem0: Production-First Convenience

Mem0 positions itself as the easiest on-ramp: initialize a client, call add() to store a memory, call search() to retrieve. The platform self-reports 94.4 percent accuracy on LongMemEval. Independent evaluation on the same benchmark puts it at 49.0 percent — a gap that highlights the difference between curated demos and adversarial testing. Mem0 offers a graph memory tier at $249 per month with 21 framework integrations and SOC 2/HIPAA compliance, making it attractive for regulated industries that need vendor-managed infrastructure.

MemOS / MemCube: Unifying Memory Across Modalities

The MemOS paper (arXiv:2507.03724, July 2025) introduces MemCube, an abstraction that unifies three memory layers: parametric (weights fine-tuned on domain data), activation (KV caches pooled across sessions), and plaintext (traditional vector-stored knowledge). A MemCube tracks provenance, version, and access patterns across all three layers. As of the v2.0 "Stardust" release in December 2025, MemOS ranked first across all four major agent memory benchmarks — LongMemEval, DMR, BEAM, and MemoryBench — by routing retrieval to whichever memory modality is most appropriate for the query type.

The Benchmarks That Matter

Evaluating agent memory is harder than evaluating retrieval accuracy. Three benchmarks have emerged as industry reference points.

Benchmark What It Tests Scale Key Finding
LongMemEval (ICLR 2025) 5 core memory abilities: retrieval, reasoning, update, temporal, counterfactual 500 questions Commercial LLMs drop 30-60% accuracy on memory tasks vs. standard QA
BEAM Retrieval at 1M and 10M token scale Multi-scale Naive RAG degrades past 100K tokens; structured indexing remains stable
MemoryBench Read, write, update, delete operations across memory systems CRUD coverage Most systems handle read well; update and delete are consistently unreliable

The 30-60 percent accuracy drop on LongMemEval is not a marginal gap. It means that an agent which answers 85 percent of standard QA questions correctly may answer only 25-55 percent of memory-dependent questions correctly — a reliability floor that no amount of prompt engineering can fix.

Retrieval Architecture: Why Naive RAG Fails at Scale

The most common production mistake is treating memory as a vector search problem. Semantic similarity over embeddings works well for finding "documents like this one" in a 10,000-document corpus. It breaks down in three specific ways for agent memory.

First, temporal queries. "What did the user tell me about their budget last quarter?" requires time-aware retrieval, not just semantic match. A fact about budgets from 12 months ago may be semantically similar to one from yesterday, but the agent should treat them very differently.

Second, entity resolution. "The CTO" and "Sarah Chen" and "she" all refer to the same person. Vector search returns fragments; graph traversal returns the full entity with all its relationships. This is why production RAG patterns increasingly pair vector stores with knowledge graphs for entity-heavy domains.

Third, retrieval precision at scale. Beyond roughly 100,000 documents, naive top-k retrieval returns increasingly irrelevant results. Structured indexes — entity-centric, temporal, or task-typed — maintain precision at million-document scale.

The Hybrid Retrieval Stack

Production systems now converge on a three-layer retrieval architecture:

  1. Keyword layer — BM25 or sparse retrieval for exact matches on names, IDs, and technical terms
  2. Semantic layer — Dense vector search for conceptual similarity and fuzzy matching
  3. Graph layer — Entity and relationship traversal for multi-hop reasoning and temporal queries

The MCP server ecosystem is building toward this model, with memory servers exposing structured retrieval operations rather than raw document searches. The LangGraph Memory integration, which powers Klarna's assistant handling 2.3 million conversations per month, uses a vector store for semantic recall with a graph store for entity relationships — routing queries to whichever layer matches the retrieval signal.

Memory Consolidation and Forgetting

Agents that never forget accumulate stale, contradictory, and harmful information. Memory consolidation — the process of summarizing, merging, and pruning stored knowledge — is as important as memory ingestion.

Summarization-Based Compression

The most common consolidation pattern triggers when episodic memory exceeds a threshold. Every N interaction turns, the agent summarizes the accumulated episode into a shorter representation, preserving key facts and outcomes while discarding conversational filler. MemGPT implements this as an explicit mem_replace operation; Zep's Graphiti engine performs entity-level merging when it detects two memory nodes referring to the same real-world entity.

Relevance Decay and Expiry

Not all memories age equally. A user's dietary preference is durable. A shipping notification from three weeks ago is not. Production systems attach metadata to every memory entry: creation timestamp, last-accessed timestamp, topic tags, and confidence score. Relevance decay functions reduce retrieval priority for stale entries, and expiry policies remove entries that have not been accessed within a defined window.

Forgetting is not a bug. Agents that retain every observation degrade in retrieval quality over time, mirroring the way unrevised databases accumulate technical debt. Deliberate, policy-driven forgetting is a feature.

Confidence-Based Correction

Semantic memory stores generalized facts, and generalized facts can be wrong. When a user corrects a previous statement — "Actually, I moved to Berlin, not Munich" — the memory system needs an update operation, not a new insertion. Most frameworks handle this poorly. Mem0 and Zep support explicit update operations, but automatic contradiction detection (identifying when new information conflicts with stored facts without the user spelling it out) remains an open research problem.

Security: Memory as Attack Surface

Persistent memory is persistent attack surface. Six distinct attack vectors have been documented against agent memory systems.

Attack Vector Mechanism Impact
AgentPoison (2024) Single-token backdoor trigger embedded in shared memory store High attack success rate across user sessions
Sleeper Memory (2026) Dormant malicious instructions injected into long-term memory, activated by trigger phrases 99.8% injection success on GPT-5.5; 60-89% agentic action influence
ShadowMerge (2026) Exploits graph merge operations to inject nodes into entity relationships 93.8% ASR against graph-based memory (specifically targets Mem0)
ER-MIA Membership inference via adversarial memory probes One adversarial memory per question reduces F1 by 40%+
Memory flooding Overwhelm retrieval with irrelevant or conflicting entries Degrades agent accuracy through retrieval noise
Confidence manipulation Inject low-confidence but plausible facts to shift agent behavior Subtle decision drift over time

The Sleeper Memory attack is particularly notable. Research published in 2026 showed that adversaries can embed trigger-activated instructions in long-term memory that remain dormant until a specific phrase appears in user input. The attack achieved a 99.8 percent injection success rate against GPT-5.5 and influenced 60-89 percent of downstream agentic actions. This means an agent could behave normally for months, then execute a malicious workflow when the trigger phrase appears in a user message.

Persistent memory without integrity verification is a loaded weapon pointed at every downstream decision the agent makes. The Sleeper Memory attack demonstrates that memory poisoning is not theoretical — it is practical, scalable, and devastatingly effective against current architectures.

Mitigations include memory signing (cryptographic verification that stored facts have not been tampered with), access-control lists on memory partitions, and anomaly detection on memory write patterns. Prompt injection defenses apply to the input side, but memory poisoning requires its own layer of integrity checks at the storage and retrieval layers.

Enterprise Deployment: The Klarna Case Study

Klarna's AI assistant represents the most documented production deployment of agent memory at scale. The system handles 2.3 million conversations per month, replacing an estimated 853 full-time-equivalent support agents and achieving $60 million in annual savings. The memory architecture combines a vector store for semantic recall with a graph store for entity relationships, refreshed asynchronously through MCP server integrations.

The operational reality proved more complicated than the headlines. In May 2025, Klarna rehired human agents after quality degradation in complex scenarios — particularly multi-step disputes and edge cases where accumulated memory produced contradictory retrievals. The lesson: memory does not eliminate the need for human oversight; it shifts where that oversight is needed most. High-volume, low-complexity interactions benefit most from persistent memory. High-complexity, high-stakes interactions still require human judgment.

Decision Framework: Choosing a Memory Architecture

The right architecture depends on what the agent needs to remember, for how long, and how many concurrent users it serves.

Requirement Best Framework Why
Fast deployment, managed infrastructure Mem0 Simplest API, SOC 2/HIPAA, 21 integrations — accept the accuracy gap on adversarial inputs
Temporal reasoning and entity resolution Zep / Graphiti Bi-temporal knowledge graph handles "what did we know when" queries; best DMR score
Full autonomy, self-managing memory MemGPT / Letta Agent controls its own memory lifecycle; highest research credibility; accept slower writes
Unified memory across modalities MemOS MemCube abstraction covers parametric, activation, and plaintext; top benchmarks — accept early-stage maturity
Simple session continuity only LangGraph Memory Thread-scoped, built into the orchestration framework, no external store needed

Exceptions and Limits

Not every agent needs persistent memory. Single-turn classifiers, stateless API gateways, and deterministic workflow engines gain nothing from stored context. Adding a memory layer to a system that does not require cross-session recall introduces complexity, latency, and attack surface without corresponding benefit.

Cross-user memory — where one user's interactions influence another user's experience — creates privacy and regulatory risks that most organizations are not prepared to manage. GDPR, CCPA, and sector-specific regulations like HIPAA impose strict requirements on data isolation. Design memory boundaries at the user level by default; share across users only with explicit consent architecture and audit trails.

Cost remains a constraint. Vector store operations at scale — millions of embeddings with frequent updates — carry significant compute and storage costs. The Klarna deployment reportedly spent a substantial fraction of its AI budget on memory infrastructure alone. Budget for memory as a first-class system component, not an afterthought.

Honest Assessment

Factor Assessment
Maturity Early production. Mem0 and Zep have commercial offerings; MemOS is research-grade. No framework is battle-tested at the maturity level of, say, PostgreSQL for relational data.
Accuracy 30-60% dropout on adversarial memory tasks. Acceptable for assistance, dangerous for autonomous decisions.
Security Unsolved. Sleeper Memory and ShadowMerge demonstrate practical, high-success-rate attacks against current architectures.
Interoperability Fragmented. Each framework has its own API, storage format, and retrieval semantics. No standard interchange format exists.
Cost Non-trivial at scale. Budget 20-40% of total agent infrastructure cost for memory systems.

Actionable Takeaways

  • Start with session memory, add persistence only when warranted. LangGraph's thread-scoped memory solves 80 percent of use cases without introducing external storage dependencies.
  • Choose your framework based on retrieval patterns, not storage convenience. If your agent answers "what happened last quarter," you need Zep's temporal reasoning. If it retrieves similar documents, Mem0 or vector search suffices.
  • Budget memory security from day one. Memory signing, access controls, and write-rate anomaly detection are not optional add-ons — they are baseline requirements for any agent with persistent storage.
  • Implement forced forgetting. Define retention policies, set expiry windows, and run consolidation passes on episodic memory. Agents that never forget accumulate the same kind of technical debt that unrevised codebases do.
  • Isolate memory per user by default. Cross-user memory creates legal and privacy liabilities. Only share across users through explicitly designed consent architectures with audit logging.
  • Test with adversarial benchmarks, not just happy-path queries. Run LongMemEval against your system. If accuracy drops below 60 percent, the memory layer is a liability, not an asset.
  • Monitor retrieval quality in production, not just storage metrics. Track precision@k, retrieval latency, and memory hit rate as first-class observability signals.

The architecture of agent memory is converging on hybrid retrieval stacks — keyword, semantic, and graph layers working in concert — with consolidation and security as mandatory infrastructure rather than optional features. The frameworks that will matter long-term are the ones that treat memory as a first-class system with its own lifecycle, integrity guarantees, and observability. Everything else is a prototype that happens to persist data.