(I think. I can't actually say I fully understand this piece, but it seems to make a bit more sense with each reread.)
(I think. I can't actually say I fully understand this piece, but it seems to make a bit more sense with each reread.)
"Let's say you see me flipping a coin. It is not necessarily a fair coin. It's a biased coin, and you don't know the bias. I flip the coin nine times, and the coin comes up "heads" each time. I flip the coin a tenth time. What is the probability that it comes up heads?
If you answered "ten-elevenths, by Laplace's Rule of Succession", you are a fine scientist in ordinary environments, but you will lose money in finance."
Why is it 10/11? If we are to take the probability by induction (as the author points out shouldn't be done), it would be 9/9, no? For a non-biased coin case, taking the limit of [h/(n)] as n -> inf and where h = 0.5n would be 50%. If we have observed 9 heads in 9 flips, doesn't induction dictate the we should expect the next flip to be heads? I'm not saying I'm right... but I'd like to correct my gap in logic.
https://en.wikipedia.org/wiki/Rule_of_succession
I'll try my hand at explaining:
Let's say there is a coin you have never seen flipped. You know it might be an unfair coin, but you don't know which side it favors. All you know is that there are two possibilities: heads or tails. Intuitively, we know at this point that the best information we have is gives us a 1 in 2 chance of flipping heads.
Laplace represents this intuition by representing what we know without having ever flipped the coin as pseudexperiments. In other words, we have two implied experiments, one in which heads was flipped, and one in which tails was flipped.
You flip the coin once, and get heads. But it would be extremely premature to assume at this point that you have a coin with a probability 1 of getting heads just because that's the result you've seen. Even a completely fair coin might have this result. Knowing that a fair coin might have this result, you can still include your pseudo experiments, and treat it as if we have 3 experiments: 1 pseudo experiment with result heads, 1 pseudo experiment with result tails, and 1 real experiment with result heads. So we now say the chances of flipping heads again are 2 in 3.
After 2 heads, we have HTHH, so we say 3 in 4.
After 3 heads, we have HTHHH, so we way, 4 in 5.
After 5 heads we have HTHHHHH, so we say 6 in 7.
And so on until 9 heads we say 10 in 11.
> Why is it 10/11? If we are to take the probability by induction (as the author points out shouldn't be done), it would be 9/9, no?
No. With a completely fair coin, there's a 1 in 512 chance that you would get heads 9 times in a row, so you can't completely discount that possibility.
The derivation proving that Laplace's method is equivalent to the sum of all the probabilities of having all the possible coins and flipping 9 heads with them is complicated, and I don't think I can do a better job of explaining that than the Wikipedia article did.
If on the other hand you choose a different prior, for example "the coin may have two heads, and it's difficult to manufacture a biased coin with two different faces that still favors one side more than 90% of the time", then you might instead reach the conclusion that there's a 65% chance that the coin has two heads. But we don't usually deal with weird priors like that in intro stats.
If, as GP suggests, you know that the coin is somehow biased, you should seed your beta distribution with a non-uniform prior. That is Beta(.5, .5)[0] or something instead of Beta(1, 1). Then you'd be 84% sure, instead of 65% sure that the coin was biased at least 90% in favor of heads.
[0]: https://homepage.divms.uiowa.edu/~mbognar/applets/beta.html Play around with this, in the Beta distribution, a=success, b=failures, X is the mean of the underlying dataset, and P(X>x) is the propbality that the mean of the underlying dataset is greater than the provided value. Look at
- Beta(1,1): the uniform prior
- Beta(10, 1): Laplace's method
- Beta(.5, .5): A prior where you believe the coin is weighted, but aren't sure how much.
- Beta(.1, .1): A prior where you're very confident the coin is weighted, but unsure in which direction
- Beta(9.5, .5): The coin after 9 flips
- Beta(9.1, .1): The coin after 9 flips when you're really confident it's weighted
- Beta(8.1, 1.1): The coin after 9 flips when you're really confident it's weighted, but both sides aren't in fact the same.
Those will give you some intuition about the distribution itself.
I have never been able to wrap a good definition around my intuition of behavior once understood is no longer effective. "Anti-inductive" is very interesting crystallization of this concept. Thank you for sharing this.
Until you’ve internalised the contents of the linked article, you should refrain from investing any money. In anything.
Haha, but not everybody knows it and a lot of different opinions exist on how awful it is or what even classifies as awful.