🐟 The Genius Who Can’t Remember — The Goldfish Problem

LLMs already possess reasoning capabilities that surpass humans. They write code, create plans, and devise strategies. However, they have a critical weakness: they cannot remember yesterday’s conversation today. Once a session ends, all data resets to a blank slate. The reason engineers call this the Goldfish Problem is clear: their memory lifespan is on the level of a goldfish in a bowl.

This problem is more serious than one might think. Gartner predicted on June 25, 2025, that over 40% of agentic AI projects will be canceled by 2027. Gartner cited excessive costs, unclear ROI, and a lack of risk governance as the official reasons for these cancellations. Additionally, some industry analysts point to the “cost of rebuilding context from scratch in every session”—the absence of institutional memory—as a key failure factor.

📦 Is RAG Dead? — The Evolving Retrieval Layer

“RAG is dead” is a common discourse, but the reality is quite the opposite. What is dead is naive RAG (simple retrieval); what survives is highly tuned RAG.

The debate between long context (1M–2M tokens) and RAG has shifted from performance to issues of economics and latency. Re-reading millions of tokens for every single query is economically unrealistic. Furthermore, the increase in Time To First Token (TTFT) is unavoidable. Additionally, the “lost in the middle” phenomenon, where information in the middle of a long context is missed, remains a challenge. Therefore, vector indices still hold a powerful advantage as a “pre-computed cache.”

As of 2026, the standard RAG baseline is as follows:

• Hybrid Search: Merging dense embeddings and sparse BM25 using RRF (Reciprocal Rank Fusion, a method that combines the rankings of two search results).
• Cross-encoder Reranking: Precisely re-scoring the top retrieved candidates.
• HyDE / Multi-query: Improving search accuracy by generating hypothetical documents or expanding queries.
• Contextual Retrieval: Indexing data chunks with the context of the entire document attached.

This is no longer cutting-edge technology but the basics. And what is being built on top of these basics is the true hot topic: the Memory Layer.

🗺️ 2026 AI Memory Market Landscape

“The model is not the product; memory is the product.” The current market is proving this proposition. Dedicated memory infrastructure startups are growing rapidly, and Big Tech companies are internalizing memory in their own ways.

Dedicated Memory Startups

Three Architectural Approaches of Big Tech

Big Tech companies are internalizing memory with different philosophies.

• OpenAI (ChatGPT): Uses a background synthesis approach. It automatically curates user profiles by “dreaming” (synthesizing) conversation history. While transparent to the user, it is difficult to directly control the content of the memory.
• Anthropic (Claude): Uses file-based agentic memory. It stores memory in a client-side file system and retrieves it when needed. This is currently the most auditable form among the Big Tech approaches.
• Google (Gemini): Uses an ecosystem graph approach. It connects personal context across Google services like Gmail, Docs, and Calendar. The key advantage is that the data already resides within the Google infrastructure.

🔬 6 Technical Trends — And What We Have Already Implemented

As stated earlier, we haven’t just observed these trends from the outside; we have implemented them while facing them directly in operational environments. Below are the six core patterns of 2026 memory engineering, organized by Market Trend → Our System’s Response. These are not mere boasts, but descriptions of the actual mechanisms we use.

① Forgetting is a feature, not a bug

The market uses four levers for memory management: Importance → Merge → Decay → Eviction. A system that stores all information forever will eventually collapse under the weight of noise.

Our System’s Response — At the exact moment a memory is created, an importance score between 0 and 1 is automatically assigned. Low-importance conversational memories follow a hyperbolic decay curve, where recall weight decreases over time; once they fall below a threshold, they become candidates for removal during nightly cleanup. Conversely, high-importance or frequently recalled memories are “promoted” to escape decay and are effectively preserved permanently, while core facts are “pinned” to be completely excluded from decay or merging. The key is to control “what to forget” through rules rather than leaving it to chance. This ensures the system doesn’t get buried in noise even after running continuously for months. This is the result of applying the principles of the cognitive science “Forgetting Curve” to real-world operations.

② Sleep-time Compute

There is a reason why OpenAI’s “dreaming” and Letta’s “sleep-time compute” are gaining attention. This method involves rapidly loading episodic memory while “awake” and performing summarization and clustering during idle time. This is cost-efficient and gradually improves memory quality.

Our System’s Response — Every night at a set time, a cleanup batch we call ‘dreaming’ runs. It consists of four steps: ① Garbage collection of expired/low-importance memories → ② Merging overlapping memories → ③ Resolving temporal conflicts → ④ Promoting frequently occurring episodes to procedural memory. In this structure, we only perform cheap loading of conversations during the day and postpone costly summarization and structuring to idle time. We also included guards against over-generalization to ensure the cleanup process doesn’t distort individual facts by blurring them together. We have essentially turned “sleep-time compute”—which the market is only just beginning to present in papers—into a running operational schedule.

③ Bi-temporal Knowledge Graphs

The bi-temporal approach presented in the Zep/Graphiti paper separates two timelines: “Valid Time” (when it was true in reality) and “Transaction Time” (when the agent became aware of it). Without this separation, models suffer from “temporal hallucinations,” where they mistake past facts for current ones.

Our System’s Response — Every memory stores “when the event occurred” and “when it was recorded” separately. This allows us to reconstruct the exact state of knowledge we had at a specific point in time through as_of retrieval (time-travel retrieval). When a fact changes, the new fact is recorded in the graph in a “superseding” relationship to the old one, preventing the model from mistaking “something that was true in the past but is false now” for current truth.

④ Deterministic Conflict Resolution

Research shows that LLMs are vulnerable when deciding which of two pieces of information is more recent. Therefore, leaving freshness judgment to an LLM’s intuition easily leads to errors. The latest approach involves bypassing the LLM’s judgment and processing updates based on timestamps.

Our System’s Response — If a new memory overlaps with an existing memory above a certain cosine similarity threshold, it is flagged as a duplicate or conflict. In such cases, we determine which is more recent using timestamps rather than the LLM’s intuition. The reason we do not leave freshness judgment to the model is simple—that specific judgment is one of the LLM’s weakest points. Thus, we intentionally handle that part with deterministic code.

⑤ Separation of Episodic, Semantic, and Procedural Memory

Among the three memory types proposed by LangMem, the core is Procedural Memory. This technology compresses verified multi-step action patterns into “skills,” allowing them to be executed immediately when similar situations arise.

Our System’s Response — We maintain procedural memory that is recalled based on task context (situation/trigger). By registering multi-step procedures that worked in the past as “standing rules,” they emerge as “standing constraints” that must be followed when a similar situation occurs, allowing for immediate application. Furthermore, we filter the scope of application for each bot to prevent procedures intended for one specific agent from interfering with another.

⑥ Multi-scope · Multi-agent Memory

mem0’s decision to separate scopes into user, session, and agent levels is a crucial one. The scope of memory must be separated to prevent information contamination.

Our System’s Response — We operate independent memory banks for each agent. Each bot’s experiences, observations, and facts are stored separately and cross-referenced only when necessary. This is a structural measure to prevent an agent’s assumptions from leaking into another agent’s “facts.” Additionally, we categorize even the same event into types—world (external facts), experience (personal experience), and observation (perceptions)—to ensure that verified facts and subjective observations do not get mixed up.

⚠️ Do Not Blindly Trust Vendor Benchmarks

There is a caveat when looking at performance metrics in the memory market. Many of the famous “mem0 vs Zep” comparison figures come from research led by the vendors themselves, such as arXiv:2504.19413. One must recognize that these results may reflect the vendor’s perspective.

There are also limitations in the evaluation methods themselves. Frameworks like LongMemEval primarily measure Recall@K (the ratio of correct answers among the top K retrieved items). However, this method has the paradox of yielding high scores even if redundant or contradictory information is stored. In real-world operations, this increases noise and degrades quality, even if the metrics look excellent.

We verify performance directly through our own Golden Set-based evaluation harness and nightly regression tests. While we use vendor figures for reference, we verify our own numbers ourselves.

Memory Poisoning — A New Security Threat

Persistent memory serves as a new vector for security attacks. Unlike prompt injection, which disappears when a session ends, Memory Poisoning turns corrupted memory into a permanent backdoor. False information survives into subsequent sessions, clouding the model’s judgment.

The industry has only just begun discussing defenses against this issue. We have secured resilience against contamination through designs that track data provenance, remove duplicates, and prioritize the most recent information in case of conflicts.

Privacy and Cost — The Cloud Paradox

Persistent memory is, by nature, a massive repository of personal information. Cloud-based memory services incur per-seat costs, and all data is transmitted to external servers. The more sensitive the work, the higher the risk.

We aim for self-hosting and operation within LANs. Data does not leave the premises, and sensitive processing is delegated to local LLMs. This is the approach we have chosen for enterprise environments. Additionally, we have designed our system to maintain high accuracy even in processing Korean and HWP documents, areas where global vendors are often vulnerable.

📚 References

• Gartner: Over 40% of Agentic AI Projects Will Be Canceled by End of 2027 (2025-06-25)
• TechCrunch: mem0 raises $24M from YC, Peak XV and Basis Set (2025-10-28)
• TechCrunch: Letta emerges from stealth with $10M seed (2024-09-23)
• arXiv:2501.13956 — Zep: A Temporal Knowledge Graph Architecture for Agent Memory
• arXiv:2410.10813 — LongMemEval benchmark (ICLR 2025)
• arXiv:2504.19413 — Mem0 performance evaluation paper (Vendor-led research)

✅ Conclusion

AI engineering in 2026 has shifted from “which model to use” to “how an agent will remember and forget.” While RAG remains important, it is not enough on its own. A memory layer that manages state is an absolute necessity on top of it.

Technologies in the market are converging in one direction: importance-based forgetting, sleep-time integration, bi-temporal graphs, deterministic conflict resolution, procedural memory, and multi-scope separation. These six are the core of 2026 memory engineering.

If you are designing an agent, I recommend asking this question before choosing a model: “What will this agent remember, when will it forget, and how will it update?” The depth of this design determines the true performance of the agent.