Introduction to Statistical Learning Theory | Hacker News Reader

Introduction to Statistical Learning Theory

stat.cmu.edu

25 pointsdangoldin7 comments

IMHO, if you want to learn Statistical/Machine Learning, theory is the wrong way to start. The best way to start is to program a few of the simplest algorithms (linear regression, perceptron, K-Means) on your own and see them run on some sample datasets available out there:

http://archive.ics.uci.edu/ml/

and only then hit the theory.

If you happen to be much better at math than programming, starting with the theory could be much easier. This survey is the best introduction I have found. The alternative being to read Vapnik's "The Nature of Statistical Learning Theory", which is more longer and much more "Russian".

Andrew Ng's notes are an excellent (non-theory) introduction to general machine learning.

http://www.stanford.edu/class/cs229/materials.html

Andrew Ng's notes are excellent but I think it'll be wrong to characterize them as "non-theory" since by lecture 8, he starts going into learning theory stuff such as empirical risk minimization, structural risk minimization, VC dimensions..etc.

Even in his other notes, when he derives algorithms for regressions, SVMs, PCA..etc, they always include a healthy does of theory and math.

In fact, I think this approach to learning the subject is far better than simply "implementing a few common algorithms" since you get a much better grasp of what the algorithm is doing and you will be able to apply them much much better once you understand some of the theory behind it.

On the content of the cs229 itself, I think the course is overall excellent. His problems sets are good too since they are hard enough to be challenging yet its not completely impossible to solve them. However, the course is a little heavy on SVMs ( like.. 5 lectures on them? ) and also leaves out some of the more interesting topics in Machine learning such as graphical models, bayesian nets..etc. But, I'm really glad that they made this course available since there isn't an equivalent undergraduate course in machine learning at my school.

I can't tell you how glad I was when, instead of a link to scribd, I got a link to a pdf when I clicked the topic.

Its a good read but its kind of heavy on theorems and math with lack of application / algorithms and background, which makes it dry to read.

To serve the field of statistical learning justice, "The elements of statistical learning" is an excellent book on the subject.

I was about to recommend that book.

I would recommend reading Programming Collective Intelligence (http://oreilly.com/catalog/9780596529321/). It features lots of coding examples and covers lots of topics (like recommendation systems, searching and ranking, document filtering, document grouping etc.)

It's way easier to start with it, than to learn statistics via pure theory.

Reply on news.ycombinator.com