I can't disagree with your AI doomerism perspective. I firmly believe that AI companies should buy one copy of whatever work they use for training. While this won't provide the never-ending royalty stream on copyrighted material that corporations strive for, it would foster the mindset that AI companies must pay society in some way. And I truly think that if AI companies are going to train on all the knowledge in the world, their profits should go back to everyone in the world. i.e., LLM models are a public good.
I have an almost unshakable conviction that LLM-type AI systems should become a repository of all human knowledge. When LLMs give you an answer, you should be able to ask, what are the sources behind your answer? People won't do this, but curious, wanting-to-learn people will. Which leads to one of the important questions. How do you keep people curious?