There was a time back in the years after dot com era where search servers were sold with the proprietary software loaded for enterprises to use. A google box running on your local network. Now there are advanced things like elastic and other search specific technologies that you can use.

I think LLMs might follow this market pattern where you can buy something to host yourself and then commoditization happens enough where open source solutions will also evolve to have good enough solutions.

An idea for a disrupting company would be to open source their LLM and offer support and feature development to enterprises as the paid offering, kinda like Red Hat or others doing that model. A key difference is running an LLM locally on decent sized compute is fine but it will be costly to scale on your own.