It's not that I couldn't have. I wouldn't have.
234 karma · joined November 10, 2022
Blog: https://blog.leonbecker.de
Stelae, a private WordPress editor that publishes static sites: https://stelae.eu
Code: https://github.com/beleon and https://codefloe.com/beleon
It's not that I couldn't have. I wouldn't have.
EDIT (comparison)
Very impressive, and a lot more work than it looks from the outside (the DAgger loop especially). I built something similar in June, vibeSH, a shell that hallucinates the whole computer, but from the other direction: no training, just a frontier model and a long prompt, so maybe the comparison is useful.
The granularity is different though. You model keypresses and predict which rows change, I batch keystrokes and have the model repaint the whole screen each round trip. Non interactive cli programs work very good, but interactive cli programs like vim, less and top do work, but it feels like SSH over a 1996 modem, you type a burst, wait, and the screen catches up. Yours is going to be a lot more responsive at that.
The big model buys consistency, for as long as the session fits in the context window. Files come back the way you wrote them, python output is usually right instead of guessed, nested paths stay where they are. The price is an API call per round trip and a cost that grows with the session (and it's not 208 MB and doesnt run offline).
Mine offers an @ai prefix that talks to the director instead of the machine, so mid-session you can say "@ai make this a haunted Solaris machine from 1996" and the filesystem, the motd and the error messages all change to match, and the directives stick.
Your hardest problem and mine are the same one I think. For you it's a 360M model losing track of state, for me it's the context window filling up, after which the box gets amnesia and old files have quietly rewritten themselves. I decided early not to fight that completely and made it canon (the box has bad memory), your DAgger setup is an actual fix for it, which I don't have.
I'm still a little worried that I'll stop posting on my blog soon, but it's actually been going better than expected. Rooting for Ben to keep posting.
One clarification: vibeSH does not run commands or inspect your machine. The model only sees the terminal transcript, and the transcript is the fake machine's state.
Anyway, soldering as a service is nothing to worry about so you're good either way.
So I'd reframe the thesis a bit: AI didn't destroy the moat, it moved it. Saving the customer setup time is no longer worth paying for, taking operational liability for the customer still is. SaaS that only offer the former are indeed in trouble, and honestly founders should verify that their product doesn't have this issue.