Agent Knowledge Needs More Than Just RAG

When people talk about “knowledge for agents,” it almost always collapses into RAG. But knowledge can, and should—be treated as a first-class runtime tool, not just a context filler.

In most implementations, knowledge lives in a vector database. Agents query it through a search_knowledge tool, often hybrid (keyword + semantic) with re-ranking. That’s essentially “Agentic RAG,” and it works well enough for a lot of use cases.

But once you think of knowledge as runtime-accessible memory, other patterns open up:

1. Dynamic Instructions Most serious agent use cases can’t just dump all instructions into the system prompt. Take a Text2SQL agent: it needs table schemas, column names, data types, and common query templates. You don’t bake all that into the core prompt—you store it and let the agent query the relevant slice when needed.

That’s dynamic instructions: system prompt as RAM, knowledge base as disk. Core logic in RAM, path-specific instructions pulled from disk at runtime.

2. Adaptive Learning Still experimental, but worth considering. The agent monitors its own conversations, flags successes/failures, and updates the knowledge base in the background. If it repeatedly fails on a domain-specific question, it can either suggest new entries or prompt a human admin to add them. Over time, the knowledge base evolves in response to real usage.

This is harder than it sounds (classification, trust, governance), but if solved, it gives agents a feedback-driven memory loop instead of a static RAG pipeline.

Bonus idea: pair the knowledge base with a scratchpad. Let the agent run iterative, multi-step searches over knowledge, not just one-shot retrieval. That’s where “deep research” style behaviors start to emerge.

I would love to know if you have tried dynamic instructions or adaptive learning?