For the same reason I don't want to carry a big heavy DSLR around with me, because there's a limited number of devices I want to lug around.
I ditched the DSLR years ago (yes, it took great photos, but not enough to make it worth the hassle). Though I still bring a pocket sized camera with me when I travel because my smartphone still can't compete with a big sensor and an optical zoom.
While a dedicated MP3 player is small (to be honest, I didn't even know they were still made), it's just one more thing to keep track of and keep charged up.
Here's a nice long article on the details: https://www.computeraudiophile.com/ca/bits-and-bytes/an-audi...
The fact of the matter is that most small devices have audio packages that simply are not up to the task, and that includes dedicated music players.