Phantom oscillations in principal component analysis | Hacker News Reader

Phantom oscillations in principal component analysis

pnas.org

24 pointsAnon849 comments

Oh wow, didn't expect to see my paper here. Happy to answer any questions!

ATM I can only read the lead .. and then it's paywall and not yet on sci-hub :-)

Would I be right in guessing that things are only stable until the eigenvalues fall to the floor and any further dimensional reductions will easily perturb?

I think this is the preprint version https://www.biorxiv.org/content/10.1101/2023.06.20.545619v1....

Sorry, I didn't have funding to pay the open access fees. The published version is very similar to the preprint: https://www.biorxiv.org/content/10.1101/2023.06.20.545619v1 Alternatively, if you want the full version, PNAS gave me some eprints to share so just email me (link on my site in my profile) and I will send you a copy!

Not sure what you mean by "things are stable" in this context, could you elaborate?

Do you think this also applies to kernel density estimation? I've used the R program and noticed extra density peaks that I wasn't able to explain except as artifacts. I guess it could've been picking up a mixture of distributions... but the densities appeared periodic if that makes any sense?

This is something a bit different, the effect in the paper comes from the eigenvectors of the data matrix. I'm no expert on kernel density estimation, but since it is basically just convolution, I would guess the effect you are describing comes from multiplication in Fourier space.

Thanks! Makes sense.. your paper is very good. It stresses the importance of understanding the mathematical tools we use and if we don’t how it can lead us astray.

This has been understood for many years in genetics. See Novembre and Stephens, “Interpreting principal component analyses of spatial population genetic variation” (2008).

Looks like that was one of the references cited in this paper.

Reply on news.ycombinator.com