It's pretty obvious. If you don't know many people who are less wealthy than you then you have no frame of reference to consider yourself wealthy.
I'd say it's more related to the imposter syndrome. IE if I got here and I'm just ok, clearly this wasn't too hard a level to reach.
That and "well I used to work with so-and-so, and we're about equal, but they're making $800k while I'm stuck making $400k, so I'm clearly failing". Or even $2M vs $10M, etc.
Again, it's crazy, because both people are clearly actually "rich", I'm just trying to explain the mechanics of why so few actually feel that way.