> will be technically out competed by their competitors because fine tuned models are better.
I disagree that fine tuning is the way to go. We spent a large amount of effort on that path and found it to be untenable for our business cases - not from an academic standpoint, but from a practical data management/discipline standpoint. For better or worse, we don't have super clean, structured data about our business. We also aren't big enough to run a full-time data science team.
Picking targeted feature verticals and applying few-shot learning w/ narrowly-scoped, dynamic prompts seems to give us a lot more value per $$$ and unit time. For us, things like the function calling API are fine-tuning, because we can now insist that we get a certain shape of response.
I have a hard time squaring an implied, simultaneous agreement with "supermodels are generally not the right approach" and "fine tuned models are better". These ideas seem (to me) to be generally at odds with one another. Few-shot learning is still the real magic trick in my book.