An intuition for how neural nets can understand text

And just opening the first video ("One neuron can act like an if-statement") shows that this hasn't been proof-watched, as you get two branches, each with "x1" on it but then you sum them not multiply (i.e the weights are not explained at all there).

Sad because the topic is super useful.

Edit: the audio levels are different between the videos too.

Without the polish, why would I pay you for your tutorials when I already pay Anthropic and can ask claude to make the same level of quality?

AI has raised the bar for effort and viability, and this isn't going to meet it.

The four parts:

1. A neuron can act like an if-statement, you can build a tiny programming language into a model.

2. A bag-of-words network can codify almost any arbitrary rule; but handling sentences it has never seen requires something smarter: rules instead of look-up-tables.

3. Embeddings: how meaning gets codified at the token/word level and why that’s useful.

4. Emergent concepts: OpenAI's 2017 sentiment neuron, which grew in a model trained only to predict the next character of Amazon reviews, and how that scales up.

Apart from the hand-set weights in part 1, every number on screen comes from a real run: a training run on 60 labeled sentences, a fill-in-the-blank run on a toy corpus, real GloVe vectors, and OpenAI's released model.

Each part is a ~4-minute video, an article with interactive figures (in case you prefer reading instead of watching) and an exercise if you want to test your understanding. The videos are generated from code: the narration's timing drives the animation, the sound and the captions.

Note by "understand" (in the title) I mean in the chinese room argument style: the network builds internal features that track meaning, like that neuron. Disclosure: I'm building zairai, an AI coach, and this course is free with no sign-up. I'd love to hear where it loses you, or where an expert would refine/make any corrections.