That's funny, when I moved to Colorado was the first time I started hearing it. My wife can't hear it at all but it's so loud sometimes at night it drives me crazy. It's not in my head, it's definitely coming from a certain direction. Its very low, I can barely see it on an app I put on my phone, "Spectroid". I'm pretty sure I know what it is, the coal trains moving down the front range. The tracks are like 10 miles away but the sound is so big and low that when the weather is right it propagates for miles. If I can hear it while the train is on a 20 mile stretch of track then a new train will come before the previous fades. I'd wager that distant freight trains and a gentle breeze cause this noise for a lot of people.

I used to live right next to a busy highway, and that higher pitched white noise didn't bother me once. This unstoppable deep noise is incessantly annoying.