Most valuable paragraph in the piece:
“The model does more than expected. Each role's skill file starts with a few sentences framing the role… This lays the groundwork as the underlying LLM knows very well how a teaching hospital works, what a differential diagnosis is, why a handoff has a structure, and what it means to be a second opinion. When you tell an LLM a persona to embody, it does it very very well.”
This took OpenAI and Anthropic some time to grasp, arguably only landing in late 2025 or early 2026, when you see them start leaving out common sense instructions, and see system prompts dropping in size. They said this was thanks to newer models, but arguably it's from no longer fighting core concepts the training compressed in.
Exercise for the reader: what metaphor best captures your system?
• • •
Below the line feedback:
These kinds of lines are painful to even decide whether to read:
“In July we audited them. Twenty-five skill files, a little over 100,000 words. About 23,000 of those words, 23%, were flagged as removable without changing a single gate, command, template, or sentinel. The dominant pattern was the same rule restated in several places: six thousand words of it across seventy findings. One file was forty-seven percent redundant.”
By contrast, these kinds of lines are refreshing:
“While this merged rather recently, and we don’t have good data to point to its efficacy, early signs seem positive.”
Fascinating example of how unfortunate the mélange.