Deep Learning Course | Hacker News Reader

Deep Learning Course

fleuret.org

447 pointsTomte56 comments

Open articleView on HN

See also Stanford's YouTube channel, where they post the entire machine learning lecture series (19 videos)

https://m.youtube.com/playlist?list=PLoROMvodv4rNyWOpJg_Yh4N...

They've posted a significant volume of CS lectures if you go to their channel. They're pretty good.

This Stanford course seems really advanced and intensive

I agree. Someone mentioned there weren't prereqs, I think, but man I was totally lost in the first class.

Oh yes, there are, anyone who says there are not is fooling others, or assuming you're a CS grad.

Are you talking about the submission course or Stanford?

Because There is a list of pre-reqs for the submitted course [1] and to be honest I feel like they are the standard requirements for you to fully understand DL may except signal processing stuff that might be taken as optional.

[1] https://fleuret.org/dlc/#information

You will have to know the basics of calculus, linear algebra, and probability. A few months of study.

If you're interested in Deep Learning or any area of ML it's fairly safe to assume you have a background in linear algebra, probability, calculus and obviously some basic programming.

If you're not interested in learning these areas, it's also safe to say you aren't really interested in deep learning either. Which is not to say if you don't already know these areas you aren't interested in deep learning, but if you don't know them and are interested in deep learning you're likely already studying them.

I say this because deep learning and the vast majority of ML really just boil down to an application of these basic tools. Deep learning/ML without the linear algebra, probability theory, calculus and coding isn't really anything at all.

> background in linear algebra, probability, calculus

Curious. What does 'background' mean in this sentence. You can spend years studying just one of these in depth. How much is "enough" for ML?

The basics. 1 semester course for each.

thanks! I wonder if someone has compiled a resource with just enough math for ML.

I believe fast.ai does this - they start with you using real tools and teach you the foundation as it becomes relevant

I'd say slightly more. Maybe it's just because I attended a state school, but I think my first semester calculus class was all single variable (20 years ago now, so my memory is rusty). You really to understand gradients and jacobians for ML, which I think was calc III for me. But you can skip curl and div part I guess.

Oh, that’s right. Multivariable was Calc II for me so technically two semesters.

how much of that do you really use for ML

Not much of your calculus book will be relevant beyond the first couple of chapters, but you'll live and die by the numerical-method sword. The idea is that you need the analytic insight from the former to understand the latter.

I don't know if I really buy that, though.

I’ve taken uni level courses for all of that but it’s been… 15+ years and is entirely out of my brain. I appreciate the feedback. Sounds like I need to brush up and I appreciate your point about it being a mathematical concept at its foundation

Machine Learning is broader than Deep Learning.

The Stanford course doesn't go as deep as, say, transformers.

Any courses you’re aware of that do?

Yes the posted one contains them (fleuret.org).

Also recommended: https://karpathy.ai/zero-to-hero.html

Oh good. Sorry I thought the implication was that this one fell short (as some of the other comments seem to suggest).

Thanks!

It's actually one of the more approachable ones you'll find.

Sorry but them there the facts. This stuff is hard. Otherwise it probably would have been done in the 1950s

IMO the only reason it wasn't done in the 1950s (noting that a lot of it in fact was done) was the widespread shortage of GPUs at the time.

Those folks were not exactly morons, they were just trying to build a nuke out of leather and driftwood.

Lots of great resources listed here. But I think there is "Understanding Deep Learning" missing from the list [0]. In my opinion, Simon J.D. Prince accomplished a true feat with his book, not only through the material itself but also with the notes attached to each chapter, linking directly to advanced references (free literature review), exercises that really challenge your understanding of the material, and great notebooks with code that truly materialized the concepts learned (free exercises to give to students if you teach a DL class, but this community is probably not the targeted audience for that).

[0] https://udlbook.github.io/udlbook/

For those interested in this course, be sure to check out his Little Book of Deep Learning as well! https://fleuret.org/francois/lbdl.html

Another great resource is NYU's Deep Learning course by Yann LeCun and Alfredo Canziani that is fully available on youtube

https://atcold.github.io/NYU-DLSP20/ https://www.youtube.com/playlist?list=PLLHTzKZzVU9eaEyErdV26...

See also: Practical Deep Learning for Coders https://course.fast.ai/

When Jeremy Howard wasn't named on the top 100 AI list, it blew my mind. This course is glorious.

Instead Anthropic co-founder siblings the Amodeis made it. Couldnt help but notice that Marc Benioff who owns Time also is the Major investor(think Khosla for OpenAI) into Anthropic. So i would take "times 100" with a side of pickle.

Are there any good, in-depth courses that don't require watching videos?

The handouts and slides are there. They are fully self-contained, not mere bullet points that the lecturer then talks about.

That's fabulous! Thanks for pointing that out

https://www.deeplearningbook.org/ , PDF version: https://github.com/janishar/mit-deep-learning-book-pdf

Thank you so much!

I think this one is a bit out of date....

It looks like it covers the basics pretty well. Any pointers to alternatives?

The "Understanding Deep Learning" book covers more recent models as well: https://udlbook.github.io/udlbook/ (free PDF and Jupyter notebooks available)

IMVHO, GANs are entirely optional.

For others, https://web.stanford.edu/~jurafsky/slp3/ will take you a decent way to understanding transformer architecture.

For specifically understanding transformers, this (w/ maybe GPT-4 by your side to unpack jargon/math) might be able to get you from lay-person to understanding enough to be dangerous pretty quickly: https://sebastianraschka.com/blog/2023/llm-reading-list.html

An earlier edition of this book was the textbook for a computational linguistics course I took back in ~2002 (!), it's amazing how much has been added

https://arthurdouillard.com/deepcourse/

https://d2l.ai/

This is my favourite text! It really drove home a lot points on many architectures for me. Their math appendix is simply amazing as well!

The deep learning book is a great choice, as many have mentioned.

I've been making a course that has a little less theory, and a little more application here - https://github.com/VikParuchuri/zero_to_gpt . Videos are all optional (cover the same content as the text).

i'll throw in this set of lectures from Andrej Karpathy. The first lecture is very accessible from a beginner perspective.

https://karpathy.ai/zero-to-hero.html

Also for someone missing some of the more advanced maths?

https://youtube.com/playlist?list=PLTKMiZHVd_2KJtIXOW0zFhFfB... another entirely approachable course (if you know some Python or similar language), by Sebastian Raschka.

I kind of want to get into this as a rusty full stack developer of several years, but I have no idea how feasible it is to try to get into this field in any capacity with 6 months of study.

If you remember what derivatives are and are decent with math (and some probability), you have what it takes (!), and can pick up most of this easier than say React, in the sense that the incline of the ramp is much less significant, but it will take more time overall (so more of a marathon than a sprint).

François, thanks for this.

I noticed on the prerequisites page

> basics in signal processing (Fourier transform, wavelets).

Are wavelets really basics in signal processing? We definitely didn't cover this my signals and systems class in EE in either grad school or undergrad.

So many options. I just started andrew ng coursera last week. What is the difference between all these free options.

I followed this course in person a few years ago. I highly recommend it.

I (we?) would welcome any details from you on how to assess it (and the many alternatives that pop up in response)?

Is there a course that includes the math pre-requisites?

These Deep Learning (and ML) courses are turning into the equivalent of productivity tools. Many are very high quality but won't turn you into a ML/DL expert. The core issue is you need to be spend the time to complete, learn and apply in your own settings. That internal drive is something beyond what these can do and that's the crux of the issue. People keep hoping some magic course will do it for them, but just like the productivity tools, there is no golden or silver bullet. Good old fashioned sit on your butt and do the work ;-).

What does this have to do with productivity tools? This is based on a university course that teaches you the fundamentals of deep learning. Of course it won't turn you into a ML/DL expert, that's not the point. Anyone completing this course on their own in their free time definitely has the internal drive that you say is lacking so I honestly don't get your comment at all.

Reply on news.ycombinator.com