I wanted to understand how LLMs really work, so I’m building Modelmathics (https://modelmathics.com) and doing every lesson myself as I go.

You start from an empty Python file and write the model yourself: a tokenizer, a bigram model, attention, a Transformer. At the end, published open weights run on your own code. Tests check each step before you move on.

Try it: https://try.modelmathics.com (or ssh try.modelmathics.com)

The rest runs in mm, a terminal app, with your code in files on your machine. Stage 1 (11 lessons) is live as a free private beta.

Feedback is very welcome.