I can't agree with you. If you work for a company that makes negative assumptions about you because of your gender/race, you'll have a worse experience than when they make positive assumptions.
It's about things like when you bring up an idea in a meeting, and it's totally ignored, and the guy across the table brings it up and it's suddenly genius. It's people acting like you always need your hand held even though you're a senior dev. Stuff like that.
While it's true that "white dude" isn't a singular set of traits... what it really comes down, most of all, is what your direct manager assumes about you. They could be biased for/against you for any number of reasons (like what college you went to, your accent, etc). But being biased based on your gender/race is probably one of the most common and strongest.
Furthermore, people who see things the way they do - who don't see their behavior as biased or abnormal - tend to clump together, creating a discriminatory culture. People discriminated against by it tend to leave, so it can be self-reinforcing. Those involved in it don't see a problem - someone they expected to be a bad programmer couldn't handle it and left, from their point of view.
Of course, you can have a terrible culture that sucks for everyone, too. Been there, done that. But there absolutely are situations where certain people/groups are favored and others are not.