The Human Regression Ensemble | Hacker News Reader

The Human Regression Ensemble

justindomke.wordpress.com

53 pointsjustindomke5 comments

That's really cool! I really liked the method for drawing and averaging predictions.

One point though:

> What’s the point of this? It’s just that machine learning isn’t magic. For simple problems, it doesn’t fundamentally give you anything better than you can get just from common sense.

I think that really depends on the dimensionality of the data and how many truly important features there are. All the examples have a single feature (and single target which is common), so it's easy to visualize in 2d. Two input features are still feasible, though it's harder to draw a manifold for the predictions, but personally for 3+ features I think it would be really tricky. You could split it to all possible pairs and draw predictions for each pair, but this way you could miss important interactions between features.

In higher dimensions, you could do a series of 1d regressions, something like this:

1. Regress x1 against y. Call your curve f1(x1).

2. Regress x2 against (y - f1(x1)). Call your curve f2(x2).

3. Regress x3 against (y - f1(x1) - f2(x2)). Call your curve f3(x3).

4. etc.

You'd have to use your computer to read the curves and recompute the residuals each iteration, but there still isn't any algorithmic machine learning.

I suspect this would work pretty well, but it would struggle when there are strong interactions between the different input variables.

There's an entire literature on this: clinical versus statistical [actuarial] prediction.

It generally finds that statistical/algorithmic methods have the edge, although there's some evidence that experts can perform as well in certain limited situations.

There's lots of explanations for the phenomenon that have been discussed.

Still interesting to see in this context.

Unless I'm misreading something, it would have been interesting to do this with a group of people (e.g., get predictions from 5 people and then average them or something--but would that then be nonhuman?)

Hey all! Together with Stan Dehaene, we recently published a series of studies showing that, indeed, humans have sophisticated abilities at performing mental regression on scatterplots. Also, our data show that they do not minimize the vertical distance of the points to the fit but, instead, the orthogonal one (as in Deming regression). Here's the link to our publication: https://www.sciencedirect.com/science/article/pii/S001002852...

Very cool. Important work imho.

Reply on news.ycombinator.com