A Course in Machine Learning
ciml.info
ciml.info
One thing I didn't see covered in depth here was feature engineering, which is the process of preparing your raw data for the machine learning algorithms. They cover it briefly in the chapter on "Practical Considerations", but anyone looking to apply ML in the real world should look into feature engineering more on their own.
One resource I recommend (and I am biased), is a python library for automated feature engineering called, Featuretools (https://github.com/featuretools/featuretools/). It can help when your raw data is still too granular for modeling or comprised of multiple tables. We have several demos you can run yourself to apply it to real datasets here: https://www.featuretools.com/demos.
In terms of neural nets, people have more or less done the hard work for you. Typically you'll want to take an empirically well-performing network like VGG, ResNet, Inception, etc and then re-train the top few layers. There is ongoing work in the field to try and train the structure of the network as well.
If you look at Kaggle, the vast majority of winners do so using ensembles. In a recent example - Iceberg or Ship - the winning team used a bit of feature engineering and apparently over 100 ensembled CNNs:
> We started with a CNN pipeline of 100+ diverse models which included architectures such as customized CNNs, VGG, mini-GoogLeNet, several different Densenet variations, and others. Some of these included inc_angle after the fully connected layers and others didn’t.
https://www.kaggle.com/c/statoil-iceberg-classifier-challeng...
It's an ugly kitchen sink approach, but it works.
most practitioners start with the simplest possible learner, then gradually, and thoughtfully, increase model complexity while paying attention to bias/variance. this is far from a "kitchen sink" approach.
It's a nice theory, and it works intuitively with models where you can ramp up complexity easily (like neural nets). It's less obvious if you have a "simple" problem that might be solved with a number of techniques. In that situation I don't see why you would be criticised for trying say any of SVM, random forest, logistic regression, naive bayes and comparing. Pretty much the only way you can categorically say that your method is better than others is by trying those other methods.
The simple approach actually came up in the iceberg challenge. The winning team won because they bothered to plot the data. It turned out that the satellite incidence angle was sufficient to segment a large number of icebergs with basically 100% reliability. So they simply thresholded that angle range and trained a bunch of models to solve the more complicated case when there might be a ship.
[0]: https://www.manning.com/books/deep-learning-with-python
But this is a good news/bad news kind of thing. Bad news, you need some statistics education to make a sense of what your computer is telling you. Good news, that mathematics isn't "high-grade" mathematics involving integrals and other stuff (as far as I can see anyway).
So what I am doing as a person who is trying to convert himself into a Data scientist is I am taking a basic udemy course (https://www.udemy.com/python-for-data-science-and-machine-le...) to give me the basics and a book on statistics to properly grok what I am seeing on the screen. On this part I am following a different book than its peers though, so it might not be for you but I like the story-telling aspect of it (https://www.amazon.com/Adventure-Statistics-Reality-Enigma/d...)
After I finish those I'll move to Kirill Eremenko's a-z courses on Machine Learning, AI and Deep Learning. I've found that even though they teach cool tricks and some basics before going in detail, that basics part didn't contain enough information for me as a new student. So I feel if you have some proper background in stats and python data analysis you can skip the parts I mentioned and go straight to a-z courses.
Also Kirill Eremenko has 38 courses listed. In what order should one take them?
Wondering if those ML/AI specialists can't find themselves a job and start writing courses -.-
Lots of people learn a lot of different ways. The more courses available for a single topic increases the likelihood that a learner will find a course that matches their learning style.
Writing courses is also a great way for someone to internally reinforce what they learned, and expose what they need to know more about.
Yes, I googled about other subjects as well. Sometimes I'm very surprised how many tutorials and materials is out there, even on very narrow topics.
In the university ages I tried to build a self made telescope. I found a lot of tutorials and books on the subject, event how to create self-made parabolic mirrors, Internet is f*kin awesome -.-
I just wondered if people with good AI/ML knowledge can find jobs.
Overall, web development & front-end jobs dwarf machine learning jobs on Stack Overflow.
Apparently 30% of Web Devs have < 5 years experience. Time to start specializing in some other domain.
Yet these jobs will still require you to do outrageous things during the interviews, like deriving the full backpropagation formulas for a 3-layer MLP network, or explain some esoteric issue with vanishing gradients or offer from memory a bunch of time complexity info about the SVM fitting algorithm, etc.
They require exponentially more impressive knowledge about machine learning in the interview than what the job experience will actually offer once you’re hired. Most positions will fundamentally make your skills atrophy.
As a result, a lot of people resort to writing blog posts or courses about their experiences implementing toy models, studying trade-offs between approaches, analyzing publicly available data sets.
In part it’s to help pad a resume and look relevant for getting hired. In part it’s to build or exercise skills that the person’s day job won’t actually enable them to use. And in part to try to get your name out there and associated with a hyped up field.
I was only responding to the parent comment regarding why random blog posts, articles, and course materials seem so widespread and constant in machine learning overall.
I deny that any part of pointing out the hypocrisy, inefficiency and myopia of tech hiring in general, including specific parts in machine learning, is "whining."
Rather, speaking clearly about how ridiculous it is happens to be progress, however small, toward correcting glaring social wrongs in our myopic industry.
And finally, none of this is in connection to my own job. I manage a team of machine learning engineers in a large enterprise company. From my own experiences earlier in my career, from some of the battles I have had to fight to protect my current team members, and from some of the horror stories my team members and job candidates I've interviewed have shared with me, my general certainty that our industry gets it completely backwards has been informed.
Between this topic and the abject failure of open-plan offices, I cannot think of any more worthwhile tech industry topics to continuously and unrelentingly "whine" about, and feel that such "whining" is in fact a civic responsibility for any mature critical thinker, and indeed I am quite proud of it, and hope to keep it up with ebullience.
Great work!