Comment by CuriouslyC | Hacker News Reader

Comment by CuriouslyC | Hacker News Reader

Parent Full thread

You can model the optimal thing to learn mathematically, by framing learning as question asking, and using information and decision theory.

Any time you ask a question, the answer will update your internal distribution of potential future (or present unknown) states of the world. The information gain of a question is the reduction in entropy of your internal future world state distribution. You can assign a value to this information gain using a loss function, which will tell you the expected loss by making the maximum likelihood bet about the future state of the world given your current knowledge of the world. The difference in expected losses is the "value" of the information.

To bring this into the realm of the practical, if there is a payout for knowledge, you should choose to learn things that minimize the chance that you will make a costly bad decision, with an eye to how probable those outcomes are. If there is no payout, you should choose to learn things that will produce the greatest information gain, which is in areas where you are currently very ignorant.

TLDR: Choose to learn things that will help you avoid likely catastrophes when learning for profit, learn a little bit about a lot of very different things when learning for fun.

She goes into this a bit in the video and notes that humans form beliefs more quickly than would be appropriate given such probability distribution model.

The thing is whether the this "optimal learning approach" is actually optimal or not depends on the world and the actual, not hypothetical distributions of futures (something we can't claim to have a complete model of). Human do extremely well in situation that AI and robots don't, so I'd say the jury is a bit out here.

> more quickly than would be appropriate given such probability distribution model.

As humans we have much sharper priors over hypothesis space than what we can easily model, which probably explains this discrepancy.

Well, our priors aren't easily expressible in terms of simple probability distributions. They could probably be modeled fairly well by gaussian process priors though.

The video claims the explanation is different. They experiments around things where humans have no priors and the humans still form beliefs more quickly than is justified by a Bayesian model - and they don't necessarily formulate correct beliefs.

The argument of the video is humans form beliefs to facilitate information exploration. In this context, any belief can be better than none.

The impression from the discussion I get is that human beliefs and behaviors tend to differentiate - people often have slightly different ideas about everything, "what is a bowl" was one example. People pick-up beliefs easily and change beliefs as they go along - as long as they have feedback.

This apparently works for groups of hunter-gathers and even for people driving cars but less well for people using the Internet to decide whether to vaccinate their children.

> where humans have no priors

In a Bayesian model there's no such thing as having no priors, that's the problem with arguing against Bayesian human reasoning with a model that can't capture the richness of human priors (which means modelling all relevant knowledge and intuition, including innate human instincts). And human priors include very strongly-held ones like "the world is basically comprehensible, governed by rules that we can discover and understand." We cannot prove that, but to the extent that we are wrong about it, all cogitation is useless, so we assume it.

Our beliefs about "what is a bowl" include that it is an instrumental concept created by other agents similar to us in order to facilitate communication. This justifies very strong priors that it will be a simple concept and easy to generalize from small numbers of examples, at least for us. All this just by virtue of being a common word. So I don't see any way to argue that human behaviour is non-Bayesian here unless one ignores relevant prior information or ignores the decision theory question "what is the consequence of being wrong about what a bowl is".

Is there a reference or book you can point to re: decision and information theory?

"Information theory, inference and learning algorithms" is a good (and free) introduction.

Reply on news.ycombinator.com