Hacker News Reader

Comment by waterside81 | Hacker News Reader

I work at a company that sells applied machine learning services, so I'd like to add a few more tricks to machine learning:

1) Have lots of data

2) Accept the possibility that your problem domain cannot be generalized.

I always find, whether in academic literature or in message boards, a desire to fit every round peg into a square hole. The reality of real world data is that sometimes, it's just a 50/50 coin toss. This might be because the features that really indicate some sort of pattern can't be defined or they can and the data can't be reliably retrieved, or the humans running things have a poor understanding of the problem domain to start with.

TL;DR: There's no magic

View on HN

My experience with real world (but still academic) data has been that there is lots of magic---feature selection to be specific.

(I'm not disagreeing, just referring to a different kind of "magic")

Everything else matters, but when your ML doesn't work it's 100% a feature selection problem. Which usually means it's 99% a problem of getting lots of domain expertise jammed up against a lot of ML experience and mathematical understanding. It's also a bear.

Agree I too found feature extraction and selection one of the vital components that helps to improve scores in lot of ML competitions hosted at Kaggle

The way 80% of real real world (non academic) data mining problems are solved:

1. Feature selection.

2. intelligent data massage. Real world data has usually noise that humans can easily identify as irrelevant or erroneous.

3. logit regression.

Starting with simple, well understood algorithms first should be the second lesson after knowing about validation sets. In those cases where they are not enough, they set the baseline for comparison against other algorithms.

Yeah I agree both with you and the ancestor. To get good results you basically have to provide the solution to the algorithm through great features. Then the choice of algorithm is not so important, as all the hard work is done in representation. So the algorithms are not magic, you have to drop the solution in front of them and all they do is find the optimal parameters for some trivial low dimensional separation manifold.

I would add 4. Ensemble methods. Having a few models helps to generalize the data fairly well.

That's a good point.

Reply on news.ycombinator.com