Furthermore, models which fine-tune LLMs are still dependent on the base model's quality. Having a much higher quality base model is still a competitive advantage in scenarios where generalizability is an important aspect of the use case.
Thus far, Google has failed to integrate LLMs into their products in a way that adds value. But they do have advantages which could be used to gain a competitive lead: - Their crawling infrastructure could allow their to generate better training datasets, and update models more quickly. - Their TPU hardware could allow them to train and fine-tune models more quickly. - Their excellent research divisions could give them a head start with novel architectures.
If Google utilizes those advantages, they could develop a moat in the future. OpenAI has access to great researchers, and good crawl data through Bing, but it seems plausible to me that 2 or 3 companies in this space could develop sizeable moats which smaller competitors can't overcome.