Hacker News Reader

I feel like ensembles are just so much easier to work with — and can be incredibly accurate given you take enough time to fine tune the parameters and provide the right features. Most ML problems that I deal with fit really well with Gradient Boosting, and it's awesome to be able to see the breakdown of how decision trees are voting.

View on HN

Using dropout with deep neural networks is an (extremely cheap) way to gain many of the benefits of using an ensemble. If you want a great overview of the dropout technique, watch this tech talk by Hinton: http://www.youtube.com/watch?v=DleXA5ADG78

Also, the recent maxout algorithm (from Montreal group, authors of Theano) that got state of the art results on several datasets is essentially just an algorithm designed to do particularly well with dropout (as far as I understand it).

You can use Neural Networks in an ensemble, and it works quite well since both ANNs and Decision Trees are highly unstable and prone to overfitting. It does significantly lower your visibility into how the algorithm works though.

Reply on news.ycombinator.com