The "bitter lesson" is from a paradigm where high quality data seemed infinite, and so we just need more and more compute to gobble up the data.

Not really.