simedw

very good question, but during training it gets the actually characters as input, and then * as what gets rendered, so I think it should understand the difference.

It's only in the models context.

That's a very cool idea, maybe a bit expensive to scale.

Exactly, I wonder if I'm not hitting the right WebGPU path for Firefox?

Thank you.

The base model already have a good grasp of language, so that man pages look like coherent sentences. It also knows python and similar language well, which means that things like 1+5 or print(["hi" for _ in range(5)]) work.

I did try to train from scratch, but it was much weaker (also added a lot of aux losses).

The first part of the context is the window width:

``` BOS SCREEN_START 32 96 ```

I trained it on three different resolutions, but it should kind of work with in between ones too.

That's very cool project. I read this blog post a few days ago about one way to look at memory encoding in models https://alexwang.ai/posts/associative-memory/ that you might find relevant

Normal only keypresses invoke the next ”frame”. There is a sleep token that the runner could use to indicate that time is moving.

I haven’t run extensive tests yet, but larger models also get better at memory.

Thank for posting it, happy to answer any questions!

It shows a good improvement on my COCO object detection benchmark: 0.504 mAP zero-shot.

Gemini 3 Flash was around 0.403, 2.5 Pro was around 0.34.

For some reason, in almost every test I did with Gemini 3.5 Live, it classified my Swedish (native) as Dutch. There are similarities between the languages, but still.

It’s pretty good at understanding my broken Mandarin, though.

This was neat, well done!

The only alternative version of Go I’ve played before was magnetic Go, where all the rules are the same except that, along the axis lines where you place a piece, like colours attract and opposite colours repel.

On Voronoi, I did feel like the continuous space made it really hard to judge whether there was enough room, whether the opponent could get in, etc. Maybe it would work better if it were discretized a bit? Or maybe I just need to play it more.

Luckily I have access to 4x RTX 4090s, so I didn’t have to pay cloud GPU prices directly. If I had, it probably would have added up quite a bit given how many training runs and experiments I ended up doing.

The biggest speed improvement came from changing the note representation when I switched to compound note events: roughly 5× fewer autoregressive passes per note.

For the current model I’m using Core ML, which optimizes the kernels the first time you run it. I haven’t actually spent that much time tuning performance beyond that.

Yes, some kind of planning step is on my TODO list. Another thing I want to try is generating a few continuations in parallel, picking the one that looks best, and then continuing from there. Maybe the picking could be automatic.

I can probably squeeze out quite a bit more than 100 notes/sec as well. I haven’t spent much time optimizing inference yet.

For DPO I only had around 700 preference examples, so not much data at all. That took about 12 minutes to train on a single GPU.

Pretraining was obviously a a lot slower, the 125M model took roughly half a day.

If the attribute describes the current note, I would first try adding it as another field/head on the note event. For example:

  NOTE(
    pitch,
    delta_onset,
    duration,
    attack_velocity,
    release_velocity,
    ...
  )

More global things might be better modelled as a control event:

  NOTE(...)
  | CONTROL_SUSTAIN(delta_onset, value)
  | CONTROL_TEMPO(delta_onset, bpm)
  | CONTROL_PROGRAM(delta_onset, instrument)

The harder part might actually be finding enough good training data with all of those attributes represented consistently.

Thank you.

Yes, I think I’ve gotten it to roughly a GPT-2 level: good enough to share, but with a lot of room left to improve. I think adding some kind of bar/measure token might help with rhythm, and perhaps some form of longer-term planning for the overall composition.

Nice package, not only is using words more token-efficient [saving time and money], but weaker models are also less likely to make mistakes when providing the key, at least in my tests.

That said, for `createAliasMap`, don't you think you could create a deterministic mapping from and to UUIDs <-> word chains? That way, no additional state would be needed. [Might require fairly long word chains...]

Cool project!

I noticed that if you go from training to watch and then back, the training temporarily drop significantly in score.

No offense, but are you a bot?

Agreed, it almost feels like we have a visual processing unit with special “opcodes” for operations like depth matching and pattern repetition.

The generator first needs a depth map, and then derives the repeating pattern from that. A normal RGB image would be far too noisy; the fine texture variations would break the repetition needed for the brain to fuse the patterns correctly.

I think this speaks for itself:

  simedw ~  $ claude -p "random number between 1 and 10" 
  7
  simedw ~  $ claude -p "random number between 1 and 10"
  7
  simedw ~  $ claude -p "random number between 1 and 10"
  7
  simedw ~  $ claude -p "random number between 1 and 10"
  7

Great suggestin, added a toggle to see pinyin.

Thank for the great feedback!

I have just added sandhi support, please let me know if it's working better.

Hi, thanks for the feedback. The 了 issue was a bug on the JavaScript side; that should be fixed (training did thankfully handle it correctly).

The other two are probably things that could be fixed with a bigger and more varied dataset.

It’s fairly sensitive to background noise at the moment. I’m planning to train an improved version with stronger data augmentation, including background noise.

For accents, I’ve mostly tested with a few friends so far. I’m wondering whether region should be a parameter, because training on all dialects might make the system too lax.

Thank you.

I had a quick look at Farsi datasets, and there seem to be a few options. That said, written Farsi doesn’t include short vowels… so can you derive pronunciation from the text using rules?