It's really not. For redbook, fs/2 is 22 kHz and while human hearing maxes at 20 kHz, it's not a hard cutoff: combine our low sensibility to high frequencies (cf ISO 226), the average listener's hearing not going much further than 18 in reality and frequency masking by other simultaneous notes and the small aliasing/imaging issues near fs/2 aren't a real problem.
But the real important factor rarely mentioned is material: the amount of music with meaningful content at a meaningful volume at those frequencies is statistically negligible.
Hi Res audio is snake oil designed to sell the same thing multiple times, period.