The Bitter Lesson makes no distinction between large general models and small specialized models, assuming they are both trained from data.

The Bitter Lesson is that human-written rule models will always lose out to trained models in the end.

Which is the way it is in life. You need a specialist. when you need to see a doctor, an engineer, a builder, a lawyer, etc, etc the world of AI will be the same. You need a specialist, a model that can focus on exactly what you need and not be cluttered up with what you don’t need.

That might mean that spending all that money on that giant master of none model might lose out to the much smaller specialist model, uh-oh…