WorldTUI: I trained a 360M model to pretend to be a terminal

Wondering if would make any difference in results, which could be impactful given your observation that "Self-correction was one of the more effective training changes"

I see a "Commit" button, but I don't see any checkout, I guess that is what happens automatically? Although that panel says "Reconstruction", not "Checkout"... You might want to do a pass after the AI created the initial version, and make it "humanly consistent" or similar, as right now it seems a bit incoherent.

> Run the JS code before and after a commit

How am I suppose to "run the JS code", manually writing it in my local JS console? Or is this just "Click the green button in the UI" but expressed in the way only a LLM would write it?

Just to point out your site text appears too small on mobile (no mobile view) and it doesn't have a direct link to the repo. If either is by choice then that's fine of course.

This isn't suprising. After all, you can use 2 neurons to model a NAND gate, and you can build up a CPU from NAND gates only.

https://nandgame.com/

You don't even need transformers to model a CPU using neural nets. The only problem becomes training them, which becomes very tricky from end to end, especially from shell command to output because the gradients are often going to reverse quite a bit due to binary nature, and the recurrence of memory outputs back into the input (we still don't know how to accurately train RNNS)

With LLMS, uou have to do something like have the LLM forget the KV caches after every invocation and instead rely on injected context of a state, which it then also outputs at the end of the terminal command, and then train it on that. That way given enough parameters which translates to resolution, and some command preprocessing (like splitting a complex chained shell commands in a single line), you can guarantee that for every possible input you get accurate output and state change. So instead of an efficient RNN, you basically end getting a much wider lookup table of sorts.

Please elaborate.

I always wanted to put it behind an open SSH server as a "fake" honeypot of sorts, see what attackers and script kiddies do with it. But I guess by now someone's tried that already..

1. Train a smaller (<10M-50M) model. Lower accuracy is fine; I want latency <10ms.

2. Then create a "reactive" version of `ssh` that gives you the perception of zero latency over shitty networks and high latency ssh connections across the world. Basically, the local screen should respond instantly to my keystrokes (including how arrow, tab, etc. esc keys respond in various TUI apps), and then correct itself when the server actually responds.

``` BOS SCREEN_START 32 96 ```

I trained it on three different resolutions, but it should kind of work with in between ones too.

https://www.youtube.com/watch?v=z3pV6FHvcgM

Time is an input too. If nothing is pressed, the model gets a WAIT token with a duration instead of a key, which is how snake keeps moving on its own.

There's a good reason why the UI for Excel, Photoshop and Word has hardly changed and remains consistent for years and that has worked for billions of people.

Many HN readers have forgotten the Windows 8 UI disaster already.

[1] https://docs.cowrie.org/en/latest/LLM.html

The base model already have a good grasp of language, so that man pages look like coherent sentences. It also knows python and similar language well, which means that things like 1+5 or print(["hi" for _ in range(5)]) work.

I did try to train from scratch, but it was much weaker (also added a lot of aux losses).

I also assume that in some way, in order to produce correct code without having to do a lot of but fixes, the larger models have gained implicit to execution engines or at keast predictors in their weights.

There are also several projects that have trained VMs into layers. Or embedded them in various ways.

There are two major potential advantages if you can really make it work well. One of them is potentially huge speed improvements by iterating inside the accelerator without deciding and running external programs. The other is being self-contained, eliminating the need for most of the harness complexity.

Another reason I think this is likely to become a real thing is that it seems with hardware improvements with flash and within a few years fully in-memory compute, we are going to get much larger, faster and smarter models. So they will be able to more easily absorb the complexity of emulating systems with more fidelity, as well as operate more quickly.

You can also imagine that as models get larger and faster, it is a natural next step in training. The top models now automatically run the software the generate and test it thoroughly. They do not need to be told to do this. By feeding enough of those parts of the sessions back into a large enough model to train on, the system gains the ability to predict the outcome without having to run the code. So it's the core part of a training pipeline already built into recent models.

There also might be some convergence with continuous learning. Because altering weights based on some token context seems like it could be similar to keeping a program resident. And maybe the continuous learning part provides the "compilation" step.