The problem is that a lot of content today is mixed so that effects like explosions and gunshots are LOUD, whispers are quiet, and dialog is normal.
It only works if you're watching in a room that's acoustically quiet, like a professional recording studio. Once your heater / air conditioner or other appliance turns on, it drowns out everything but the loudest parts of the mix.
Otherwise, the problem is that you probably don't want to listen to ear-splitting gunshots and explosions, then turn it down to a normal volume, only to make the dialog and whispers unintelligible. I hit this problem a lot watching TV after the kids go to bed.