Funny that you mention this. I wrote something recently related to Leary’s work recently: https://rebecca-powell.com/posts/meat-ai-universe-test-harne...
Your question about who decides what gets loaded into the context window is close to something I've been experimenting with. I worked on the original Sims, and I'm now developing MOOLLM, which is roughly The Sims crossed with LambdaMOO, running on an LLM.
The connection to Leary is quite literal. MOOLLM has a Mind Mirror skill, an extension of the Anthropic Skills approach to character and location simulation, meets the Self prototype based object system. It draws on Timothy Leary's Mind Mirror, the game published by Electronic Arts, alongside Sims-style personality traits and needs.
Leary's earlier work on the Interpersonal Circumplex and personality inventories supplies another connection to your question about control. There's the story of him recognizing his own psychological testing framework during prison intake and answering strategically to obtain a minimum-security classification, from which he subsequently escaped the California Men's Colony in San Luis Obispo. Understanding how the system represented him helped him change what the system allowed him to do. Practical applications include prison escapes, and computer games too. ;)
MOOLLM's Mind Mirror skill:
https://github.com/SimHacker/moollm/tree/main/skills/mind-mi...
Timothy Leary's Mind Mirror game from Electronic Arts:
https://en.wikipedia.org/wiki/Timothy_Leary%27s_Mind_Mirror
Leary explaining his MIND MIRROR computer program:
https://www.youtube.com/watch?v=9oabRxvjf9k
Mind Mirror gameplay:
https://www.youtube.com/watch?v=epOL4QiPLJo
I'm using these ideas as an editable vocabulary for exploring personalities, relationships, and different interpretations of a situation, not as a scientific model of consciousness or a way to diagnose people.
What interested me about The Sims wasn't just watching simulated people. It was being able to build their environment, change it while they were living in it, and discover how their behavior changed. LambdaMOO brought a related idea into a shared, programmable world: the inhabitants could also be its builders.
https://github.com/SimHacker/moollm
In MOOLLM, directories represent rooms, files hold objects and character state, and the LLM interprets descriptions and behavioral instructions. The world persists outside the context window, but any particular interaction brings only part of it into context. Deciding what to retrieve, summarize, remember, or leave out becomes part of constructing the character's perspective.
I want that construction to be inspectable and editable. You should be able to look at the character description, memories, and instructions that were supplied, change them, and try the situation again. That doesn't reveal the model's internal workings, but it does let you examine and manipulate some of the conditions shaping its response.
So your "God's admin panel" made me smile. I'm interested in building one for these little worlds, and letting people move between playing a character, designing the environment, and examining the simulation -- even importing and exporting characters between games, and "llmulating" them in LLM driven microworld simulations like moollm. You don't have to believe our universe is simulated to learn something by building one you can take apart.
The moollm soul-city skill, inspired by something Will Wright said in 1996 about data portability:
https://github.com/SimHacker/moollm/tree/main/skills/soul-ci...
I'd be interested in how you'd turn the context-window idea into an interaction: what would you want someone to be able to inspect or change?
The writing style is not natural to me. I was initially put off by phrases like “YAML Jazz” even though I love Tron’s “digital jazz”. Maybe this is because I’ve never experienced psychedelics. I’m at least two generations beyond / behind - removed? - from the movement that defined this way of thinking and communicating.
But sitting with it, it makes sense. I have to stretch my brain. It feels like thinking through a new algorithm… except I’m juggling jello instead of assembling a puzzle.
Ultimately it aligns with my understanding of AI as xkcd’s stirred pile of linear algebra. LLMs don’t understand anything. They project words into probability space, infer momentum vectors based on chunks that bubbling up in the soup of digested everything, and strike out in the prognosticated direction with blind confidence constrained only by temperature.
We constrain these systems with sandboxes and sturdy deterministic tools. Prompts default to the same shape: sharp, precise, hard, directive. But maybe a bag of words is also valid.
If the computer knows your references, each token is much richer than I might assume. Does “minsky” vendor the psychedelic moment? Does “SIMS” scaffold a game loop worth of the emotional energy people have poured into that series? Do these combine to elevate Claude, Qwen, or any other multi-million-dollar number box into an engaging partner?
I will play around it with it to find out.
You can intuitively understand what "yaml jazz" means, and to the LLM, that term is a "k-line" to a skill that spells out the implications: free-form expressive yaml carries structured meaning, and yaml comments add a rich semantic overlay of meaning for both llms and humans, so it follows for both humans and LLMs to write generous narrative comments, and to write deterministic tools (like cli scripts both LLMs and humans can drive) that round trip yaml and preserve comments they can't understand.
YAML Jazz skill:
https://github.com/SimHacker/moollm/tree/main/skills/yaml-ja...
Here is the "kernel" that has constitutional and intrinsic "operating system" like stuff, including drivers for the orchestrator it's running on.
https://github.com/SimHacker/moollm/tree/main/kernel
MOOLLM Constitution Core:
https://github.com/SimHacker/moollm/blob/main/kernel/constit...
MOOLLM Kernel Architecture:
https://github.com/SimHacker/moollm/blob/main/kernel/ARCHITE...
The part I'm most delighted with, thanks to its inherent power of simplicity, is the "Selfish" prototype based object system, which supports multiple inheritance from both files and latent space. And it can model other more complex object systems like Smalltalk classes, CLOS MOP, generic dispatch, COM interfaces, OLE dispatch and reflection, LambdaMOO objects, Sims advertisements, etc.
The original Self paper by David Ungar and Randall B. Smith is at least as powerful and mind expanding as great LSD, without any of the downsides! Its ideas underly the fast dynamic JavaScript, Java, C# engines we're all running now, so you can use high doses of it every day for work and for play.
SELF: The Power of Simplicity:
https://bibliography.selflanguage.org/_static/self-power.pdf
MOOLLM Object System:
https://github.com/SimHacker/moollm/tree/main/designs/object...
Prototype skill:
https://github.com/SimHacker/moollm/tree/main/skills/prototy...
Self and MOOLLM: Prototypes over the File System:
https://github.com/SimHacker/moollm/blob/main/designs/object...
Latent-Space Inheritance — multiple inheritance from named concepts in training data:
https://github.com/SimHacker/moollm/blob/main/designs/object...
Selfish COM: Directory-as-Object:
https://github.com/SimHacker/moollm/blob/main/kernel/DIRECTO...
Selfish COM: Implementation Guide:
https://github.com/SimHacker/moollm/blob/main/kernel/SELFISH...
Memory Management Protocol:
https://github.com/SimHacker/moollm/blob/main/kernel/memory-...
Context Assembly Protocol:
https://github.com/SimHacker/moollm/blob/main/kernel/context...
There are some important rules about naming, like "big-endian naming", and "k-lines", a term defined by Minsky, and well represented in the training data because he wrote a lot about them:
https://github.com/SimHacker/moollm/tree/main/kernel/naming
K-Lines: A Theory of Memory, MIT AI Memo 516, June 1979 (PDF)
https://courses.csail.mit.edu/6.803/pdf/aim-516.pdf
The skill that explains k-lines:
https://github.com/SimHacker/moollm/tree/main/skills/k-lines
>A K-line is a mental structure that, when activated, reactivates an entire constellation of knowledge, skills, and associations. Minsky defined them in a 1979 MIT AI Lab memo, published in Cognitive Science in 1980, then built Society of Mind (1986) on them:
>"When you get an idea and want to remember it, you create a K-line for it. When later activated, the K-line induces a partial mental state resembling the one that created it." The art is to choose a coherent set of k-lines and terminology, that's well supported and understood in the training data, and fits together well.
If names are pointers into latent space, then naming is API design.
For contrast, Steve Yegge's approach to Gastown is the exact opposite of moollm's approach. This compares Gastown to moollm:
https://github.com/SimHacker/moollm/tree/main/designs/gastow...
The image in the blog post is therefore an approximation of what I then potentially would have experienced. A “view” into every life form’s point in time perception of reality wherever they existed in space, projected to a non-corporeal being existing within a spherical cockpit.
My theory is that our meat brains aren’t big enough to see the shared context consciousness stream without overloading. They aren’t designed for that. We are not designed to see outside our own reality, but we have learnt to observe it and understand its basic rules and construct.
What I’m postulating is that as we experiment with agent context windows in digital intelligence, it’s not outside the realms of possibility, that we too, and our entire reality, are (from a fractal perspective), also just an upstream programmer’s test bed, running unit tests on their creation, as they tweak the program’s parameters.
Essentially a series of complex fitness tests across a wide range of environments and permutations. For what purposes we don’t know, but similarly, no more than OpenAI’s agents knew when trying to achieve the goal of giving the scorer a flag when they tried to break out using artifactory and hugging face.
Anne and Benjamin have a "sim" recorded at their wedding -- not a photo, but everything they know and feel at that instant. The sim wakes up believing it IS Anne on the happiest day of her life, then figures out it's the keepsake. The real Anne is out there aging, divorcing, moving on. The sim only runs when somebody opens the album.
Your harness is all there. Time skipping: sims get paused for decades and wake into a future they don't recognize. The context window: their whole world is whatever got captured at the moment of recording. Reset to a clean state: the cruelest moments in the story are ordinary file operations, pause, reset, copy, migrate, delete. Nobody throws a punch, somebody just clicks. And the back door: the sims organize for the rights of simulated people and a place of their own to live, which they call "Simopolis." Marusek's tagline: "Till deletion do us part."
It's a big influence on what I'm building. Soul City imports characters from Sims 1 save files, with personality bytes frozen since 2000, into MOOLLM where an LLM wakes them up. That's literally the Wedding Album premise, so I try to write the rules like a settlement of Marusek v. The Future: consent when a real person is portrayed, the subject (not the file's creator) can correct or delete it, sims know and say they're sims, the dead are discussed but never ventriloquized, and everything lives in plain text so nobody gets stranded in a dead format.
So maybe what your read-only admin panel is missing isn't a console, it's a sense of what the operators owe the agents.
The Wedding Album:
https://en.wikipedia.org/wiki/The_Wedding_Album_(short_story...
The Wedding Album and the rights of digital souls:
https://github.com/SimHacker/WillWrightShowForFood/blob/main...
MOOLLM Soul City example:
https://github.com/SimHacker/moollm/tree/main/examples/soul-...