The Bitter Lesson only applies when you don't know the function you are modeling (e.g. perfect chess strategy or text-to-speech).

On the contrary, neural networks will never give you a better way to convert to the frequency domain than the Fourier Transform. At best, they might approximate it.