Learning Machine Learning: A beginner's journey

I think what Andrew Ng would say is that without a rigorous statistical background, you will be limited in your ability to use ML, and you will certainly be more liable to blow your foot off by using it improperly. That being said, in a subset of cases, you may be able to achieve non-trivial insights through the techniques he teaches in the course.

So how I would rephrase your assertion is that a hacker can probably get a lot more out of ML techniques if they are willing to learn the math underlying them.

> Oftentimes, you do need a solid theoretical/mathematical background. Most people seems to approach ML like they approach programming tools or libraries - learn just enough to get job done and move on.

I've been coming across this on HN front page and it's worrysome to an extent.

If you don't work on your fundamentals, you end up simply memorising the algorithms and basic applications. Much like the neurons discussed for deep learning, we need to create rich relationships between our own to ensure we retain and can apply the knowledge in the long term, which starts with laying a foundation of fundamentals.

For brushing up on those fundamentals, I recommend Bertsekas & Tsitsiklis : Introduction to Probability. Theory supported with lots of examples, as well as comparing the theory to "intuition" and why it is much more effective to apply the theory.

Frankly I regret spending time understanding all the math, instead of working for some company, munging through their data and applying some black box stuff. The hype is quite bad IMO.

But 99.9% of people are not trying to compete with Google or Facebook. They are looking for basic insights.

I do agree that ML isn't and shouldn't be "easy". But we inevitably and thankfully get abstracted away from the inner workings of things over time.

A great resource specifically tailored to those that don't have a especially strong grasp of probability and statistics is Grokking Deep Learning

https://iamtrask.github.io/2016/08/17/grokking-deep-learning...

This is not entirely true. Most of their advantage comes from the corpus of data. Of course, I'm not discounting the fact that they're pioneers in the field, but at this stage data is their competitive advantage (hence they open sourced Tensorflow.)

I feel the present state of ML libraries or even distributed system libraries not being a black-box solution and not the "just works" type is a growing pain, and will be evolve into something more accessible/robust in the future. The whole point of it being a "layer of abstraction" is that you don't need to know the details.

I think writing some algorithms and using them to solve problems provides great motivation for the math. In particular, the math will explain why certain approaches did and did not work. Without the hacking that material can get a bit dry.

Developers everywhere use Paxos without even knowing it, much less having read Lamport's papers, because they're building on top of solid tools that use Paxos (or Raft or what have you). This is more true at Google and Facebook than anywhere.

Same goes for ML. You can study the theory, and you can learn to apply it. In the field's nascency you basically need to understand the theory in order to apply anything, but eventually robust tools are built upon which developers can build systems without having "studied these things for years (by PhD)".

Not entirely true. Google & FB have orders of magnitude more data than flipkart. You can have the smartest ML people on the planet churning out the most clever, advanced ML algos & models, but without enough data, its not going to be useful & effective.

I recently attended slashn[0], flipkart's annual technical conference, and spoke to a bunch of their ML folks. They have masters & phd degrees in ML from IITs, IISc and are as smart as they come.

Sure, flipkart doesnt have marquee names of the likes of yann lecun, andrew ng, but i wouldnt doubt the ML talent Flipkart has

0. https://slashn.flipkart.net/ ps : I dont work for flipkart, but have friends who work in ML teams

- Too few labeled / garbage labeled data (70000 digits? How about only 1000 complex class objects?)

- Obscure bugs in custom implementation (yeah my custom layer works and gradient is correct... or wait why it diverges after 10k iteration? hmm).

- Timing/RAM constraints (it should segment an image under 10ms on Jetson TX1, well, good luck with GoogLeNet)

This type of comment is often made in machine learning (ML) related submissions.

The pre-req list is long: calculus, linear algebra, stats, probability, numerical methods (for optimization, linear algebra, maybe interpolation), etc. BUT, you don't really need to go through the entirety of each subject for ML. For example, in calculus, you probably only need to focus on the aspects necessary for optimization, rather than integral techniques, convergence of sequences, etc. The trouble is that it is difficult to know which subtopics of each subject are worth spending time on unless you already know machine learning (or you have the luxury of someone with experience guiding you).

The latter difficulty is compounded by the fact that there seems to be many more resources (at least posted as popular submission on the web) for learning neural nets or learning some specific framework to implement neural networks, than to learn the mathematical and statistical foundations of ML. This is fine -- neural nets are a popular and powerful model, and people like to work on something tangible to get acquainted with a topic.

I wonder if people might enjoy a well-written textbook covering the basic math for ML -- something like, "All the math you missed (but need to know for machine learning)" [1]. I might enjoy working on such an ebook if there was desire for one, but my time is pretty limited (like most).

[1]: https://www.amazon.com/All-Mathematics-You-Missed-Graduate/d...

Currently, DL is the most powerful technique for many problem types. I think for a beginner, learning that is a safe bet, the "next thing" is likely to be an elaboration on DL. It's good if some people ignore the hype, they may come up with the next paradigm going in a completely different direction from DL. But the people doing that won't be beginners. They'll be people with similar levels of experience as Yann LeCun, Geoffrey Hinton, etc. Those well rounded people with deep theoretical knowledge will make the big breakthroughs. Beginners should start with what we know works well now, and expand out.

> [Other techniques may be] simpler and work just as good

This is a much better reason to learn other things than DL. Efficiency and lower complexity if you know the problem domain is amenable to the technique.