For those who don't know about Plato's Theory of Forms, the remarkable empirical fact is that all humans learn roughly the same concepts representing words like "dog", "house", "person", "boat", and so on, even if those people grew up in different places and never had any overlap between their observational experience. You and I may never have observed the same dog, but we still agree on what a "dog" is.

This phenomenon appears to occur for LLM learning as well, though it is less remarkable due to the fact that LLMs likely have significant overlap in their training data.

I believe this is good news for Alignment, because, as Plato pointed out, one of the most important forms is the Form of the Good - a (theoretical) universal human ideal containing notions of justice, virtue, compassion, excellence, etc. If this Form truly exists, and LLMs can learn it, it may be possible to train them to pursue it (or refuse requests that are opposed to it).