It's the natural conclusion of their beliefs in themselves as ubermensch - that the rest of us are untermensch. I'm sure many are nice on the surface, but they fundamentally don't think of us as equals - there's a good reason why they didn't go to their state schools after all.
You can see it in people like Dan Gross who named his venture fund after the eugenics research fund Pioneer Fund and called other people "NPCs" in a lecture that got big on HN a few years ago.