Hacker News Reader

Comment by rckoepke | Hacker News Reader

Mainly I take issue with the title "...: The Complete Guide". More accurately, its a very abbreviated overview. The table of contents has a fairly reasonable list of topics, but for a complete guide I'd want each section there to be at least an entire chapter of material. Instead they are 4-6 sentences per section.

Personally, I think that any comprehensive guide/textbook for machine learning, especially deep learning, should contain a chapter on the mathematics of control systems theory. The feedback/back-propagation is very similar to what EE's and ChemE's do with PID loops or Op-Amp tuning, and I feel that so much is lost academically when that topic is glossed over.

View on HN

>The feedback/back-propagation is very similar

i really wish people wouldn't say these things just to sound smart. it confuses people that don't know better. feedback loops have nothing to do with backprop, which is just a way to factorize the jacobian into matrix vector products instead of matrix matrix products. do you have some proof (and it would require a proof) that PID controllers actually compute the gradient of a function with respect to n-inputs? because that's what backprop does.

Thank you, non-sarcastically. Would edit if I could.

I'm not even sure Deep Learning is at the level where a title like "The Complete Guide" could make sense.

Deep Learning involves harnessing advanced, highly complex and rather ad-hoc algorithm to engage in systematic-but-heuristic prediction.

A complete guide would include all natural prerequisites, the common approaches, the best practices and the areas of application. But all of these are in flux as the field races ahead. Moreover, the field still needs "artists", practitioners who can figure out the black-art of training networks. So whatever it's virtues, the field today seems incomplete to me.

Yeah, pretty much everyone can benefit from a high-level grasp of phase margin and gain margin. Even in contexts where exact calculations are impossible, it's important to remember that cranking the gain too high turns your amplifier into an oscillator and that lengthening the feedback cycle reduces the gain ceiling above which this happens.

Typically, I find that people have an intuitive understanding of the possibility of overshoot but not of oscillation and almost never of the connection between feedback lag and overshoot/oscillation.

I liked the birds eye view of this article, but that title

Do you have a recommendation for a book explaining the relevant control systems concepts?

Reply on news.ycombinator.com