Memorilla squeezes an entire document collection into 16 memory tokens
Memorilla: Latent Semantic Memory for LLMs
Memorilla gives frozen LLMs a memory system that encodes documents once into reusable embeddings, then learns to retrieve query-specific information directly into the model's input embedding space as a fixed set of 16 memory tokens. Across eight benchmarks covering personalization, long-document and knowledge-intensive QA, it beats Mem0 and MemoRAG on five. On FactKG it hits 84.33% accuracy versus 80.43% for full-context inference while using 108x fewer tokens, and in TextWorld it lifts an agent's success rate from 18.6% to 24.2%.
Using only 16 memory tokens, Memorilla achieves the best performance against baselines including Mem0 and MemoRAG on five of eight benchmarks spanning personalization, long-document and knowledge-intensive question answering.