Hacker News Reader

Comment by tel | Hacker News Reader

My experience with real world (but still academic) data has been that there is lots of magic---feature selection to be specific.

(I'm not disagreeing, just referring to a different kind of "magic")

Everything else matters, but when your ML doesn't work it's 100% a feature selection problem. Which usually means it's 99% a problem of getting lots of domain expertise jammed up against a lot of ML experience and mathematical understanding. It's also a bear.

View on HN

The way 80% of real real world (non academic) data mining problems are solved:

1. Feature selection.

2. intelligent data massage. Real world data has usually noise that humans can easily identify as irrelevant or erroneous.

3. logit regression.

Starting with simple, well understood algorithms first should be the second lesson after knowing about validation sets. In those cases where they are not enough, they set the baseline for comparison against other algorithms.

Yeah I agree both with you and the ancestor. To get good results you basically have to provide the solution to the algorithm through great features. Then the choice of algorithm is not so important, as all the hard work is done in representation. So the algorithms are not magic, you have to drop the solution in front of them and all they do is find the optimal parameters for some trivial low dimensional separation manifold.

I would add 4. Ensemble methods. Having a few models helps to generalize the data fairly well.

That's a good point.

Agree I too found feature extraction and selection one of the vital components that helps to improve scores in lot of ML competitions hosted at Kaggle

Reply on news.ycombinator.com