Yeah, this is what I don't get. Nowadays you can just uninstall any Apple app you're not using so I don't see why the big fuss?
As far as I know, if you want to be able to sync music from your computer or use a third-party, non-steaming music app you must have Apple Music installed as well.