There's nothing inherently weak about the fidelity of 16-bit audio on its own. PC audio subsystems don't deliver the full dynamic range on a single audio channel by default. They reserve headroom so that they can mix additional audio sources with less risk of clipping. Audio players that let you increase the volume beyond 100% are just letting you use the full range.
None of this is relevant to a real, dedicated music playback system that doesn't contain a digital mixer. You can't hear noise at -96dB. Your amplifier will swamp that with it's own internal noise sources. In the 80s the audiophools loved to complain that CDs were too quiet because their beloved LP noise was supposed to be desirable for some whack reason.