Agent memory as a file format
Cal Paterson proposes a radically simpler approach to AI agent memory: instead of complex pipelines, treat memory as a portable file format called 'memoryfield'. It consists of Markdown pages with optional YAML frontmatter and a SQLite vector index for semantic search. This design avoids the pitfalls of existing systems—lock-in, complexity, and context stripping—by letting agents write memories directly in prose, jump directly to relevant pages via semantic search, and scale with model improvements. The format is open, interchangeable, and transport-agnostic, with tooling and a spec provided.
Show me your flowcharts and conceal your tables, and I shall continue to be mystified. Show me your tables, and I won’t usually need your flowcharts; they’ll be obvious.
- dataviz1000
Does anyone else not use memory?
I find once there is one poisoned line of text it negatively affects everything else downstream. Instead, I use a temp/ folder with documents and use different files for different agents and models. Then I have to constantly prune and delete the files. Any information that can be extrapolated is just noise which negatively affects the agent. If you have a definition of a database structure and it has been implemented, that information should not be contained in any text document -- it is noise, will drift, and be impossible to debug why the agent keeps producing undesired behavior.
I have a ~/Projects folder. For example, I use Playwright with Chrome DevTools Protocol in order to do performance testing and leak detection. There is a script that handles this. My prompt is "Search ~/Projects for perf testing with CDP and Playwright and implement here". Point being, if I need anything I point to a resource or ask to search a resource and it will find it quick and, most importantly, tends to improve it every iteration.
If I was in an institution, I would have a repository and would rather just point the resource and say use that than have memory of it locally.
- JohnMakin
> Markdown "pages", with
> (optional) YAML frontmatter and
> (optional) SQLite vector index for semantic search
This is basically exactly what I use in a MCP API I built and it works pretty well. Can be enriched further if you use a storage system like S3 and take advantage of metadata.
"harness managed" memory is utter garbage, I am convinced, and I disable it immediately. The major problem being over time it degrades and sneaks in conflicting or outright false information. Then one day you'll swear it's drunk, and every time I got to this state and investigated, auto managed memory was always the problem.
- morelandjs
Was ready to write something snarky because this is essentially RAG, but I think the author is getting at some subtle details which are seemingly important.
- memory systems are a specific type of knowledge base where you generate all the documents. You might as well generate them to be less than your embedding token limit to obviate the need for chunking.
- embedding models are getting better and are no longer just semantic averaging.
- small models are getting dirt cheap, making parallel reads cost manageable
What they describe is sort of the simplest architecture that takes advantage of these observations. I believe them when they say it works well.
I do suspect though that things like keyword lookup will completely fail if every memory is just a vector. Hence why something like Typesense hybrid search can still be useful.
- Avijit_Thawani
"Irrelevant material is simply never surfaced by the semantic search."
thats quite optimistic. there's lots of "memory" or past chats with agents that should be suppressed and forgotten because they were looking in the wrong place or were eventually proven wrong. yet semantically they'd look very relevant to a future search. thats why you shouldn't search both textbooks and scifi when trying to solve an examination.
- JustFinishedBSG
That's a whole lot of text to say "it's markdown".