Have seen an interview with someone from openai. At least they claimed, that they are far away from running out of data and they are able to generate data if needed.
If they can generate data they should use that instead of piping it through a million-dollar lossy compressor.