Hacker News Reader

Comment by phreeza | Hacker News Reader

In the general sense, is out-of-distribution learning even possible?

The best you are likely going to get is robustness wrt different environments, but the environments have to come from the same Meta-Distribution. Invariant risk minimization is an interesting research area in that direction, there is for example an adversarial formulation that seems quite elegant. I have never tried this, but I can imagine it is tricky to get to converge.

View on HN

>In the general sense, is out-of-distribution learning even possible?

Yes. Humans can learn to answer questions even when questions (test cases) don't come from the same distribution as the training set.

You need to have some amount of analytical capabilities. Analytic as the ability to form distinct and modular concepts from the input and form representations that can be composed together to generate instances that don't exist in the distribution. The low-level example of this ability could be blind source separation.

I guess it is a question of definitions, but i think it really is not possible, for the reason that the space of possible test sets includes contradictory sets. So if I ask you a simple question which is not in your training set, like "Does a Quigloth have eyes?", there can be two possible test sets, one where the answer is no, and one where the answer is yes. So a reply to this could be that questions have to be about the real world. But that is then the same "meta distribution" as I mentioned in my previous comment, and not out of distribution learning in the pure sense.

Reply on news.ycombinator.com