memU

: https://github.com/NevaMind-AI/memU Happy to hear thoughts from people building agents or infra in this space.

m as natural-language statements. Embeddings are optional and used only for acceleration. - Memory Category Layer Aggregates items into readable, theme-based memory files (e.g. user preferences, work logs). Frequently accessed topics stay active; low-usage content is demoted to balance speed and coverage. --- ## Memorization Bottom-up and asynchronous. Data flows from resources → items → category files without manual schemas. When capacity is reached, recently relevant memories replace the least used ones. ## Retrieval Top-down. memU searches category files first, then items, and only falls back to raw data if needed. At the item layer, it combines BM25 + embeddings to balance exact matching and semantic recall, avoiding embedding-only imprecision. Dual-mode retrieval lets applications choose between: - low-latency embedding search, or - LLM-based direct reading of memory files. ## Evolution Memory structure adapts automatically based on real usage: - Frequently accessed memories remain at the Category layer - Memories retrieved from raw data are promoted upward and linked - Organization evolves from usage patterns, not predefined rules Goal: keep relevant memories retrievable at the Category layer and minimize latency over time. --- ## A Unified Multimodal Memory Pipeline memU is a text-centered multimodal memory system. Multimodal inputs are progressively converted into interpretable text memory, while staying traceable to original data. This provides stable, high-level context for reasoning, with detailed evidence available when needed—inside a memory structure that evolves through real-world use.

but incorrect context — especially harmful for agents that act over long horizons. memU takes a different approach. Instead of trying to make embedding search smarter, we ask: what should not be retrieved via embeddings at all? Retrieval in memU starts at a Memory Category Layer: - memory is organized into semantically stable categories - each category is stored as a readable Markdown file - these files act as long-term, canonical memory When a query arrives, the LLM reads the relevant memory files directly, using semantic understanding rather than vector similarity. Only when this layer is insufficient does memU fall back to item-level retrieval, optionally using embeddings for speed. This design treats the LLM as what it’s increasingly good at: reading, reasoning, and maintaining structured knowledge, not just ranking vectors. Using Markdown files is deliberate — similar to ideas like `skills.md` — making memory explicit, inspectable, and stable over time. Compared to existing approaches: - [mem0](https://github.com/mem0ai/mem0) is fast and simple with classic RAG, but can struggle with temporal accuracy and precise state changes. - [Zep](https://github.com/getzep/graphiti) uses graphs, which handle structure well but add complexity and maintenance overhead. - [memU](https://github.com/NevaMind-AI/memU) uses non-embedding retrieval to address RAG’s structural limits in accuracy, stability, and long-term consistency — without replacing RAG entirely. For long-running agents, retrieval needs to provide reliable premises for reasoning, not just relevant text. In those settings, direct LLM reading over structured memory often aligns better with how models actually reason.