The number of Americans with passports is at an all time high of 48%, and some of that is simply for convenience ID for internal flights.
I'd expect that if you look at your wealth instead of your income, you probably come to a similar conclusion. If you are in the top 20% income bracket but do not feel rich, my guess is that you are probably in a lower wealth bracket (i.e. you have a house but still a huge mortgage on that house and not a lot of other assets, putting you somewhere in the top 40-50% wealth bracket.
If your wealth puts you in a similar bracket as your income (i.e. top 20%) and you still don't feel rich, this is more a case of your perception being out of whack. Top 20% in terms of wealth in a society like Germany by definition makes you a very rich person. But even our millionaire future chancellor described himself once as "middle class", so this is a common mistake to make ;-).