Hacker News Reader

Comment by CrypticShift | Hacker News Reader

Excellent! I'd love to see a script that sends a list of "descriptions" (1-100 words) to ChatGPT and directly gives you back a ready-made (embedding vectors closeness) map in a (textual) graph/chart format (like your above map or your plot https://interconnected.org/more/2023/02/in_our_time-PCA-plot...)

View on HN

It turns out that "closeness" is usually hard to visualise/explore when you're dealing with a 1,000-dimensional space... and PCA has the failures mentioned above.

It's weird -- it's locally useful to navigate, and at a high level kinda useful, but only if you squint and don't look at the problems. So I feel like a fisheye visualisation would be appropriate? That's something that I'm exploring in other projects.

I wouldn't necessarily reach for PCA. No reason to think that the first two principal components necessarily encode anything particularly interesting. If you want to lay out each point in 2D in a way which keeps similar points nearby, something like t-SNE is worth a try - visualizing embeddings is what it was invented for.

Excellent, new to me and I'll give it a go, thanks!

I gravitate to PCA for terrible reasons (undergrad so it's what I think of first) and like you say, it's beguiling yet disappointing, the components rarely have any human meaning.

I'd suggest trying t-SNE [1] instead; you'll be losing almost all of the variance by projecting onto the first two eigenvectors produced by PCA.

[1] http://karpathy.github.io/2014/07/02/visualizing-top-tweeps-...

TSNE or UMAP (as others have mentioned) is good but also take a look at the tensorflow projector. You can host your own and/or pass custom data to it.

The word game of 'semantle' (https://semantle.com/) is a nice way to get an intuitive grasp of how unintuitive the closeness in a highdimensional space is, as you're required to guess a word based on it's semantic similarity (according to classic word2vec metric) to your previous guesses.

Reply on news.ycombinator.com