GPT-4 is pretty clearly not just public data + public papers. It's a lot of optimizations and other things on top of what is itself a lot of optimizations and other things that led to GPT-3.

We're going to have to wait and see if OpenAI has a moat. You could have argued in 2001 that Google had no real moat. All they were doing was indexing public data with an algorithm they'd already published in a research paper, after all. But it turned out to be a deep space with an endless potential for optimizations and improvements, combined with the need for great expertise in scalable computations and ad optimization that other companies just weren't able to catch up with even after trying hard for years.