The latter is definitely on OpenAIs radar. They're even hiring specifically for this problem: https://openai.com/careers/research-scientist-superalignment
But where's the team protecting us from a future where we have no more shared experience or discussion of media anymore because everyone is watching their own personalized X-Men vs. White Walkers AI fanfic?