Agents don't need memory, they need documentation

I think it's going to be an incredibly common, maybe universal cycle people will go through working with AI until they realize it doesn't work long term.

Why something exists, and how it connects to the outside world may be documented in comments, but more often than not it isn't.

But this article argues that LLMs do better when the context is smaller — when it can understand the totality of the task with as little context as possible. And so having correct API-level docs is greatly advantageous. Anecdotally, this rings true to me — when the local context is good and clear, the LLM writes code matching my intent even when my prompt is sloppy and poorly specified.

Rejoice! The LLM will write the docs for you, relieving you of most of the work.

However without intervention, it will do too much and record absurdly verbose docs (similar to how an LLM will relentlessly refactor your code until you instruct it to move in minimal, incremental changesets). You will still need to edit down what the LLM generates.

We have heard this nonsense from the "we don't need to write comments, code should be self-documenting" types for decades. It was wrong in that context, and it is wrong in this one.

Code tells you how a system works. It does not tell you why it works that way. That is what memory is for. It exists so that your AI does not keep undoing past decisions when it writes or refactors code.

I liked this advice when humans wrote code. Though even then I'd urge people to write meaningful commit messages that capture the "why" of what they did, so no one tramples their intent by mistake.

But not sure it works in an age where most code is LLM-generated. Especially if that code is not even reviewed by humans (irresponsible or not, it's happening), and commit messages are also generated by AI. I think something is needed to separate "what did the human operator intend" from what the agent went and built.

I do agree that this gets way overengineered. My approach has been more or less what you stopped doing though - committing all our timestamped "plan/implementation docs" and "investigation docs" that document what the user wanted + empirical findings, and making all prior session transcripts searchable. It's seemed mostly helpful? For whatever reason I haven't run into many staleness problems so far.

When it is not, it has to be reasoned out and does not work well for documentation.

Other things that code and tests alone do not capture well:

- promises (as in Promise Theory) made to other parties. Claude already has PT in its training data.

- Constraints-inducing-properites, as in Roy Fielding / Christopher Alexander. While tests, and property testing can capture properties, there is no formal connection to the constraints that induces fhem. By constraints, I am not talking about business requirements and business value — those are better understood through Promise Theory. I am talking about things like at-least-once delivery or total ordering (from append-only constraint). Claude already has Fielding’s dissertation and Alexander’s works and ideas in its training data.

- grammars, as in pattern panguages (not just patterns) a la Alexander / Fielding are also not captured in code alone. These tell both humans ans AI how to extend a pattern, and how to identify anti-patterns (when they violate a constraint-inducing-property)

- LLMs are trained with many different worldviews and bounded contexts at the same time, and is very capable of translating across it. However, these need to be soelled out, otherwise it would talk in whatever it infers

Specifications written for the exact way components are wited together run into that stale doc problem. Although it takes much more human attention and token burn to describe things in terms of pattern language and promise theory, it becomes easier over time. The actual implementation plan tends to fall out more cleanly when all those other stuff are at least considered. This is where I have been spending most of my time.

In my codebase it is difficult to get agreement on comments and documentation so rather than rely on it I adapted. One of the first things I did when I succumbed to agentic development was to point codex at the code and ask it to generate a high level description of where important files, such as our public API, reside, what the hierarchy is, what the code does, etc. In my case, this level of documentation is fairly static if I avoid implementation details. So now I have a handful of agent files in my tree and it seems to save quite a few tokens and improve my results. I frequently have other devs ask me how I get such good results when doing agentic reviews of their changes(always my first step now before I start my human review). I also include instructions in the agents files instructing the agent to maintain the agent files if any relevant changes are made. It seems to work quite well for me.

All code is written under constraints, and most constraints live outside the code.

    You don’t need documentation […or…] memory

    The code IS the documentation

Trivial example: LLMs struggled with QA on our app. They didn’t know how to find the seeded test accounts. They would create new ones and do it wrong, or find the seeded test accounts but not know their passwords because they were encrypted, so they’d change the passwords but not tell the other agents. Shitloads of tokens burned. It was a no-brainer to just give them the credentials in a skill that gets loaded when they do QA.

You could say that’s an environment issue, not a code issue. But as far as actual code IME at a minimum we have to tell the agents our general repo structure and architecture patterns.

The days of “you are the world’s greatest Python programmer, write good code” or whatever are over (if that style of prompting even worked in the first place) and as I said less is more. But, still….

Who are your stakeholders? How are you going to communicate with them? What kinds of approvals do you need to do certain things? Etc. Human devs have long struggled with this stuff and LLMs aren't at better.

This way the primary agent only has relevant information in their context to make decisions and take actions.

Context management is still under valued imo.

Not today. IMHO Documentation is just a form of structured memory and it’s all just context. Getting that context right is a hard problem and there’s a lot of different ways to skin that cat.

I don't know, but I have a pretty standard (I think?) setup, and Claude manages to find every relevant file every time. But I've also only used Claude for greenfield projects, where "documentation is primary, and code flows from documentation" is the philosophy.

I have CLAUDE.md describe all the types of documentation files and the directory structure. And then Claude is pretty aggressive (automatically) about always inserting cross-references everywhere. So a feature description will reference the ADR's that it implements, the ADR's say what feature implements them. A code file will make reference to the "implementation design" document that describes the motivation behind which iOS elements were chosen, how the animation is defined in a particular way that doesn't break another animation, and so forth. So I've really never run into a situation where Claude failed to read a file it should have. I've been pleasantly surprised.

I would say that the one really big thing I've had to learn is to teach Claude both in CLAUDE.md and in the header of every top-level design document, that keeping documentation current and in sync is paramount. Because its default seems to be to keep history and append, e.g. by default it will take a section of a document and mark it "[DEPRECATED]" and add the new version below. So my instructions are pretty clear in having it always be aggressive in maintaining current state only, always replace rather than append. And if there's anything we want to save from the previous approach (e.g. we did it X way previously and it failed because Y), then just add that as a new short note in the new current-state text, possibly with a pointer to a commit or tag or something.

So this seems to solve both recall and staleness in my projects at least.

The only thing I still haven't found a solution for is numbering. Claude is always giving everything numbers, like F23 for feature 23. But I'm always changing the order of things, inserting new things, deleting things, so I wind up with a sequence of development work that goes in order like "Phase 9", "Phase 9b", "Phase 9e", "Phase 11", "Phase 12". I'm halfway ready to abandon numbers entirely and just start giving things names from noun collections instead, so every feature is named after an animal, every ADR is named after a kitchen implement, or something. Or just four-digit hex codes chosen at random. Curious if anyone else has found what works.

When you have knowledge distributed in markdown files; finding them puts the path/filename into context as well as some indication of document size. (If its on line 1200 or line 20). This is extremely valuable for picking what ought to be focused on next.

RAG on the other hand creates the hardest challenge for these models. It instead puts 5 ideas with the highest similarity into the context in full.

Its the difference between having to remember a set of numbers when in a crowd that's talking about stuff, and having to remember them when the crowd is shouting out random numbers. The similarity in the task makes things harder. SoTA models work despite this, but its extra-gambling while you're already gambling.

I think there is a deeper problem emerging from this sort of behavior. Even when we bother to create agent skills with there own scripts that call tools like jq a specific way to achieve a goal, AI coding assistants and agents still go way out of their way to generate ad-hoc scripts to do the most absurdly stupid tasks such as parsing output in structured language, and even remove whitespaces from a markdown file. This means AI coding assistants and coding agents treat agent skills as mere suggestions of using a alternative option that more often than not the choose to ignore.

This has a very dangerous implication: your average user is trained to develop a pavlovian reflex to authorize agents to just execute their ad-hoc scripting code with our own permissions and credentials in our systems, which includes the ability to call anything over the internet.

https://gist.github.com/cynthiateeters/6868ca26c059a3106cd93...

I’m the author of the project.

Once qwen3.8-flash-next showed up, it cañ go "forever" with dynamic context pruning.

It's fascinating for local coding.

I write README and docs by hand, thus only important stuff goes in there. If it is not worth my attention to write down, it's not worth writing down.

Smaller context, easier to consume.

Then for example you could write your own hook and convert existing code smell documentation which agents ignore into a format that works with the hook.

It doesn't even understand when its current context isn't...

See the whole issue of refining a concept, removing a few points or notions, and it will write a paragraph why a point that isn't in the text anymore when no version a human will read ever had that point in it.

The questions are designed specifically to identify the state the agent is in, in order to assign its steering. Changing this tool changes how the agent works. You just need to make the use of this tool a necessity so it does not forget to call it.

Unlike principles and memories, questions are more openended and tend sometimes to trigger the right mentality in the agent even before it gets to the policy itself. In my opinion agents get lost in local work losing the big picture, my questions jog the big picture back into attention.

1. call ssp tool, no arg, it just reads features from the repo, like most other tools -> ssp locates state and sends 4-5 questions

2. agent responds the first batch -> state is further narrowed down -> ssp sends a second batch of questions

3. agent responds again to the interview -> state gets finally pinned down -> agent gets the steering assigned

4. a log is created of this ssp session, and reflection on the log used to refine the questions and steerings in the ssp tool (name comes from State-Space Policy)

It seems you tried to reinvent instruction files. What do you think is the difference between your approach and standard tools such as instruction files, AGENTS.md, and even skills?

Do you have any resources that guided your work?

The code is the documentation. There's no need for most of this stuff.

https://github.com/skorokithakis/gnosis

This just does a pass at the end, rather than consume context all the time. I've been using it for months and it works great.

I've been bitten by agent-written ADRs. Agents carelessly add extrapolated details and speculated nice-to-haves that I never asked for, and this becomes a source of bloat that keeps coming back like a boomerang.

https://github.com/ArtRichards/docs-cli

and

https://artrichards.github.io/agent-playbook-suite/blog/

I think a lot of people are stuck on patterns that "used to work" (and I use that loosely because I really feel like LLMs are getting work done IN SPITE of using all these additional skills/plugins/etc). Go ahead, delete all that shit and try building without. Yeah, sometimes you may have to hand hold the LLM, but that hand holding is a LOT less than doing whatever this is.

This all just sounds like a rube goldberg machine you've made.

"Code is the documentation" doesn't solve the problem in my experience. Because what happens is that you still need a lot of "why" comments in the code, and then these go stale, so you still have the same problem you have to solve. And so I find that markdown documentation is a lot easier to organize and review in a structured hierarchical way in one place, than code comments sprinkled across the repo.

Legally and culinarily, they're vegetables. Botanically they're fruit.

Tomatoes are members of the nightshare family which includes tobacco, potato, and chili peppers.

They are used in popular recipes like Mexican salsa and Italian pasta sauce. Pasta sauce commonly uses the San Marzano variety. Here's a <picture> of San Marzano tomatoes from our Italy trip.

Rhett likes tomatoes in all dishes. Link only likes tomatoes if they're blended in a dish like pasta sauce or tomato soup. Frank is allergic and can't have tomatoes.

Tomato prices are up due to a bad <year> yield in <country>.

---

This is all memory. Where does it go?

That said - I personally ended up saving my plans and status in my second brain app via MCP. Having that available to me on the road when I'm in the car brainstorming with Claude over voice is a big win.

I take the article's point more directly. It's just a straighter line to have clear communication through documentation than to fuddle around with the perfect memory setup.

I don't think you need more documentation or these memory systems. The code IS the documentation. Lots of this stuff is unnecessary. It's a bunch of people doing their special rain dances and then when it happens to rain they say "I did that"

https://github.com/Clumps4Linux/SEP/blob/main/SEP-1.1.1.txt

Github Copilot had the idea of attaching memory to files, and if the file hash changes the memory is automatically dropped (not sure if they still do it). This means they are overly eager to drop stuff (even if the file change is just cosmetic), but at least they don't accumulate outdated cruft too much. (a memory can still be outdated if it was invalidated by a change in another file though)

You’re confusing, I think, memory system with llm finetuning. Completely different concepts.

I think what you are trying to say is that “give the LLMs access to information and it can figure out how to use it. Don’t get fancy about how to structure the data”.

If that’s the case it has largely turned out to be true. RAGs have fallen out of fashion.

Where does that put AGENTS.md though? Is it worth spending time to structure it or just dump it.

Every single project at work that has documentation/ADRs/etc ends up stale as fuck and it pollutes context more than it helps. Even the agent generated docs. Hell, especially the agent generated docs. It's like the LLMs can't just DELETE something, they always amend. So now there's stale details in there about something we abandoned polluting context.

The code is the documentation, none of this stuff is needed. I'm seeing WAY too many devs/engineers go down the rabbit hole of trying to build these tools and systems but not actually getting meaningful work done.

I've been doing my best to remove and delete all the documentation. I usually keep a super high level AGENTS.md/CLAUDE.md file and that's it. The llm/agent can figure it out. You might argue I waste 5 minutes and a dollar or two for the agent to "figure it out" each time, but I'm pretty certain people are wasting FAR more time and money building these things out and maintaining them in each of their projects (or NOT maintaining them and they're actively BAD for their projects).

How, if at all, do you keep the architecture or a working theory of the code in your head?

But that's one session. Isn't memory for… the next session?