Comment by macrolime | Hacker News Reader

Comment by macrolime | Hacker News Reader

I'm in the same boat kinda, but even more outdated on some parts. I had some AI specialization back in college, but that was before deep learning was even a thing, so we did stuff like self-organizing maps and evolutionary algorithms, but it wasn't really all that useful for much back then. Been following deep learning from the sidelines, but for work my AI work has been restricted to GOFAI until recently.

Some of the stuff I'm currently reading/watching or have recently

Practical Deep Learning, though it sounds like you may know this stuff already (https://course.fast.ai/)

Practical Deep learning part 2, more about diffusion models. Full course coming early next year (https://www.fast.ai/posts/part2-2022-preview.html)

Hugging Face course (https://huggingface.co/course/chapter1/1)

Diffusion models from hugging face https://huggingface.co/blog/annotated-diffusion https://huggingface.co/docs/diffusers/index

Andrej Karpathy's Neural Networks: Zero to Hero. He goes from the basics up to how GPT etc, so you can start wherever suits you (https://www.youtube.com/playlist?list=PLAqhIrjkxbuWI23v9cThs...)

3blue1brown's videos. I've found all his videos on neural networks and math worth watching, even for stuff that I already know, he sometimes has some new perspectives and nice animations.

brilliant.org. Nice math refresher and the courses there are almost like fun little games.

Have you had a look at https://nnfs.io/ ? I bought the book and am gearing up to start working through it, I would be interested to know your thoughts. Generally I want to chart a personal curriculum from data engineer to practical application of modern AI to real business problems.

I hadn't seen that book before.

Looks like it's a companion to this YouTube series that looks pretty interesting https://youtube.com/playlist?list=PLQVvvaa0QuDcjD5BAw2DxE6OF...

I will check it out for sure

I'd be interested to know what you think, it's quite a time investment to study the entire book.

The neuromatch computational neuroscience course also seems quite interesting, though maybe less of practical use.

https://compneuro.neuromatch.io/

Recent research like "Relating transformers to models and neural representations of the hippocampal formation" might make it more relevant though (https://arxiv.org/abs/2112.04035v2)

quote from the abstract of that paper: "Many deep neural network architectures loosely based on brain networks have recently been shown to replicate neural firing patterns observed in the brain. One of the most exciting and promising novel architectures, the Transformer neural network, was developed without the brain in mind. In this work, we show that transformers, when equipped with recurrent position encodings, replicate the precisely tuned spatial representations of the hippocampal formation; most notably place and grid cells. Furthermore, we show that this result is no surprise since it is closely related to current hippocampal models from neuroscience. We additionally show the transformer version offers dramatic performance gains over the neuroscience version. This work continues to bind computations of artificial and brain networks, offers a novel understanding of the hippocampal-cortical interaction, and suggests how wider cortical areas may perform complex tasks beyond current neuroscience models such as language comprehension."

Reply on news.ycombinator.com