A gentle introduction to information geometry | Hacker News Reader

A gentle introduction to information geometry

robots.ox.ac.uk

70 pointsbindidwodtj9 comments

This article is really well-written and has an excellent description of differential manifolds.

Worth a read just for that part (not implying that the rest isn't good as well).

[Edit]: and the rest is really good as well. In particular if you ever wanted to understand how space-time (4D) can be intrinsically "curved" without it being embedded in a larger dimensional space (5D), this explains it very well.

Glad to hear it. I agree it's well written. I was doing okay up until the co(ntra)variant tensor part. My brain is going to get a helluva workout trying to follow this. :-)

Technically correct reading tends toward the dense.

It takes several readings to tease out a clear understanding of each of the points in the formulae. The terseness of the expressions is daunting.

Several more readings are needed to internalize it to the point of teaching it to someone else, which is when the real grasp arrives.

Or maybe I'm just slow.

> It takes several readings to tease out a clear understanding of each of the points in the formulae. The terseness of the expressions is daunting.

I was quite pleased with the writing/my understanding of the first section. As you say, it is dense but quite readable IMO. Math tensors (as opposed to computer/ML tensors) are new to me so that will take more time.

> Several more readings are needed to internalize it to the point of teaching it to someone else, which is when the real grasp arrives.

That's part of my study process, can I explain the topic to my imaginary students. Works well for me.

> Or maybe I'm just slow.

Nothing wrong with savoring things at a leisurely pace.

Whenever I see <something> geometry, I get excited that there will be more pictures than equations (like in geometric proofs). Sadly, it was not the case here, but I learned something new anyway.

My experience from writing peer-reviewed papers in the area (roughly speaking) is that some of it gets very abstract and it's a bit difficult to illustrate all the time.

But you can do it sometimes and I agree it's helpful.

Thanks for your comment! If you have a minute, are there any intro articles or blog posts in this area that you would recommend for people lacking your mathematical maturity? (Also, I finally have an excuse to reference my favorite xkcd comic: https://imgs.xkcd.com/comics/average_familiarity.png)

So practically, how does having this understanding affect the way one thinks about certain probability distributions?

I'm not sure it affects how you might think about probability distributions in general, although it might. I think it's more helpful in thinking about inferences about distributions and models?

The page mentions one concept, for example, pertaining to uniform distributions. We usually think of an uninformative prior distribution in Bayesian inference as being uniform, which it can be, but what you probably want is a uniform prior with regard to a metric of parameter distinguishability or something like that. This will often change depend on the model, and might not be uniform with regard to the original metric of the raw parameter.

There's a lot of other uses an implications along these lines: how you compare and evaluate models and parameter estimates.

Reply on news.ycombinator.com