Hmm I'm not so sure about that; do you have any source for that? My intuition says that it's nigh impossible to find completely quiet places in daily life; there's always some noise, though it could be imperceptably quiet.
If constant sound is harmful, I'd imagine we'd all have lost our hearing by now. I reckon any harm woulc come from volume, not just simple presence.
That's assuming it's physically harmful (to the ears). If it were instead mentally harmful, what you would expect is for the incidence of mental illness to increase as life got noisier and noisier (with a lag, of course, assuming this effect takes an appreciable amount of time).
Are we saying our brain is not processing sound while we sleep? Ain't we hearing sound 24/7?