https://colab.research.google.com/github/stared/thinking-in-...
By all means, much more "from scratch".
It is a part of open-source "Thinking in Tensors, Writing in PyTorch": https://github.com/stared/thinking-in-tensors-writing-in-pyt...
This course uses python and pytorch and will have you building something that can tell 30+ breeds of cats and dogs apart in the first lesson. While the old course from Andrew ng you mentioned starts with theory and by the end gets to doing something useful, the fastai course starts with best practices and moves into the underlying theory later.
Also Andrew Ng has a newer deeplaearning.ai course on coursera that is all python and doesn’t do Octave.
https://news.ycombinator.com/item?id=21301117
I derive the normal equations in just two lines of matrix algebra, never use calculus, assume nothing about probability.
The normal equations are plenty simple to program in nearly any programming language. For the linear algebra, just use LINPACK or for a really simple approach just write your own Gauss elimination linear equations solver.
"You can see also expected value notation, were E[x] means an average of x, that is ∑N−1i=0xi/N."
I think this is not what E[x] means, i.e. going from expectation to the average (using the sum over n-1 no less) involves a bit more than notation (in this case, assumptions about the DGP). Yes I also hate posts like this, but in the interest of this being a course on mathematics in DL - wanted tollet you know.
(I can add that with "uniform measure" and "arithmetic average", but for people learning it it is too much, for people knowing that the context is obvious anyway.)