Intuition for Probability and Likelihood | Hacker News Reader

Intuition for Probability and Likelihood

notion.so

22 pointsbehnamoh9 comments

That's a posterior, not a likelihood.

And the >1 issue is when you're dealing with a probability density rather than a probability mass.

Sorry it was a typo. Corrected.

The fact that like no probability books mention linear forms forms a travesty https://en.wikipedia.org/wiki/Linear_form

Why would they? All of the probability-theoretic structure comes from the fact that a measure is not just an arbitrary one-form, and has lots of extra properties. And then if you want to do probability over arbitrary manifolds, you need to talk about differential forms in general: there's no point at which it makes sense to generalize to one-forms but no further.

Maybe "Probability via Expectation" by Peter Whittle https://link.springer.com/book/10.1007/978-1-4612-0509-8 or "Infinite Dimensional Analysis" by Charalambos Aliprantis and Kim Border https://link.springer.com/book/10.1007/3-540-29587-9?

Probability as a mathematical object is very simple: it’s the proportion of a quantity (measure) distributed over a set.

Likelihoods are a product of one or more probabilities over a subset of the set, so they don’t necessarily sum up to 1.

You are mixing up a few things. If the set contains infinitely many objects, you cannot talk about proportions anymore, because the probability of every point is zero. Also, the term "measure" has a well defined meaning in mathematics. It's better to not use it synonymous with the term quantity. Likelihoods and probabilities are both two-argument functions, where you simply change which argument is considered to vary and which one is fixed. The function f(D, theta) = Pr(D|theta) defines the probability of the data D given the parameters theta of a data-generating model. Usually, one holds theta (the model) fixed, and asks for the probability of some data D being generated by that model. But when asking what is the most probable model to have generatey already observed data D, D becomes fixed and theta variable, i.e. how likely was the data under different models? This is measured by the likelihood, L(theta) = Pr(D|theta), where we try to maximize this function over the set of all possible theta for some fixed data D.

Thanks for the response.

Could you say whether for finite sets, my statement is correct?

Also could you explain when measures and quantity are the same and when they are not?

TL;DR #1 is correct. #2: Quantity usually means just a number but isn't a maths term. A measures is a function that maps sets to numbers. A probability is a special kind of measure.

For finite sets it is correct. More specifically, the quantity you are talking about is the number 1 and it is distributed over all elements of the set such that every element gets a number between 0 and 1 (where also the borders 0 or 1 are allowed), and the numbers of all elements sum up to 1.

Quantity is not a mathematical term, but we usually use it in every day language to just denote a number. A measure, mathematically, is a function. A measure of a set A assigns _every subset_ of A to a number greater or equal 0 and has two key properties: 1) The empty set is always mapped to the number 0. 2) The union of every two subsets X and Y of A, that also don't have any element in common, is mapped to the sum of the values of the subsets X and Y, i.e. measure(X ∪ Y) = measure(X) + measure(Y).

The cool thing is: Probabilities are a special kind of measures! Remember that a measure has to assign a number for every subset of a set A? Since A is a subset of itself, a measure has to also assign A a value. A so-called probability-measure is a measure that assigns A the value 1. Now you might guess where this is going: If you want to define probabilities for something going on in the real world, you first define a set that contains all the things that can happen. E.g. if you define probabilities for a dice roll, the set A contains the 6 possible numbers, A = {1,2,3,4,5,6}. Each subset of A corresponds to a possible event. For example, the set {2,5} means "either 2 or 5 appears". How is the probability-measure P defined? If the dice is fair, we have P({1})=P({2})=P({3})=P({4})=P({5})=P({6})=1/6. From property 2), we then get P({2,5})=1/6+1/6=1/3, i.e. the probability that 2 or 5 appear is 1/3. And of course we have P({1,2,3,4,5,6}) = 1, i.e. the probability that any number appears is 1.

Sorry for the long reply, but I hope I could clarify a few things. Maths gets very fast very technical, but it's also nice to understand some key concepts.

Reply on news.ycombinator.com