>I’ve always felt there was something really interesting about the idea that someone might internalize the color blue as I see the color red.

Yes! In fact, philosophers have spent a lot of time thinking about this exact problem:

https://plato.stanford.edu/entries/qualia-inverted/

https://plato.stanford.edu/entries/qualia-knowledge/