30+ Machine Learning Resources

If you really think about it, there are so many wide ranging tools in machine learning that you simply cannot spread yourself thin enough to stay sharp and "interview-ready" on all of it.

Just off the top of my head, I can think of:

svm, decision tree, random forest, gradient boosted trees, simple ensembling, Gaussian processes, classical regression, complex treatment of standard errors, Bayesian regression, hierarchical regression, mixed effects models, structured prediction, outlier detection techniques like RANSAC or one-class SVM, agglomerative clustering, k-means, approximate nearest neighbor methods, missing data techniques, EM algorithm, non-parametric regression, locally weighted regression, regularized regression, model evaluation concepts like goodness-of-fit tests, likelihood tests, interpretation of R-squared, how these metrics change with different data collection changes or problems, nuanced understanding of the pitfalls of p-value reasoning, Bayesian p-values, classical confidence intervals, Bayesian credible intervals, logistic regression, softmax regression, ANOVA, linear discriminate analysis, graph algorithms, graph centrality measures, Page rank, min-cut and norm-cut, segmentation models, perceptron, and multi-layer perceptron, pros and cons of different activation functions, pros and cons of different momentum-based optimizers in deep learning, general backpropagation algorithm, nuances of how complex recurrent neural networks like LSTM or GRU work, exploding or vanishing gradient problems, nuanced understanding of weight initialization techniques in deep learning, batch normalization, stochastic gradient descent, nuanced understanding of learning rates, training diagnostics to detect overfitting or sample biases or convergence, how to use deep learning libraries to make customized network topologies, domain-specific topics like word or character embedding representations in natural language models, bag-of-words, language distributional assumptions, shallow neural models like fasttext and word2vec, multi-task learning, computer vision models, face detection, segmentation, inpainting, natural scene statistics, adversarial networks, adversarial noise, techniques for exploring interpretation of high-dimensional models like weights in a deep neural network, pivotal papers like AlexNet, ResNet, Inception, glorot, cyclegan, style transfer, learned data structures, dropout, etc. etc. Comvolutional neural networks and how the shapes of output layers derive from inputs and filter sizes, the usefulness of 1x1 convolutions, regular max pooling as well as global pooling layers, understanding of why max pooling is used to learn larger-scale spatial relationships and why capsule networks are meant to address that in a more rigorous but harder to train way. Reinforcement learning (both Q learning and deep reinforcement), AlphaGo, monte carlo tree search, general MCMC methods like metropolis and gibbs, and advanced ones like variational bayes and NUTS, and how to use them with STAN and pymc, and diagnostics to monitor convergence of sampled Markov chains, sampling methods like importance sampling (and in what situations it reduces error), rejection sampling, inverse CDF, basic pseudorandom number generation, sensitivity of all these things to IEEE floating point specs and precision, digital signal process basics like FFT, autocorrelation, time series modeling with ARMA, ARIMA, differencing, trade-offs vs just using indicator variables or lagged regressors in more standard models, hidden Markov Models, Kalman filters, particle filters, A/B testing, as well as working understanding of many SQL RDBMSs, Hadoop, Spark, appropriatness of GPUs for different methods and how to understand associated cost trade-offs, and working knowledge of data visualization tools to communicate about results.

... and this is what I can think of literally stream of consciousness. I am certainly still missing huge topics and application areas.

And then consider that in an interview you can be asked about any of these, about nuanced model fitting diagnostics, computational complexity, tricky riddles about how does the model change if you resample the data or remove a feature or rescale something... is it rotationally invariant, is the loss function convex, what to do about overfitting?

And then, after all that grueling preparation and intense study, followed by your unassailable interview performance where you heroically explain these algorithms in great technical detail but simultaneously with "insights" and "interpretability" that pleases the most ephemeral of fleeting political interests... after it all, you are rewarded with a job writing maintenance code for bullshit Spark jobs, or A/B testing a new purple button on the advertising page that the product manager has already decided is better and stops the A/B test early just as soon as it appears to give the desired result with statistical significance... and you start to open your mouth to mention multi-peeking bias in A/B testing, but you stop, because you remember that nobody cares.

The reality is that statistical modeling practitioners should be learning the trade-offs and nuanced evaluation details of the whole variety of possible models through their work which should (because it is value-additive to the employer) include plenty of freedom to explore modeling trade-offs and engage in work and professional development allowing you to stay current in a wide variety of statistics topics.

If anyone finds any job like that anywhere, please let me know!

Until then, I guess interviewers will keep telling us we're not good enough if we don't know all about the latest paper on arxiv that was published while we were in the bathroom just now, and if through some miracle we get the job, we'll be promptly asked to forget everything about statistics and just be ML-literate devops engineers or stats-literate Excel admins.

What a field...