Thinking that anyone with significantly more money than you are alien beings with evil tendencies is an unhealthy belief. It's both wrong and self limiting.

If I had a couple of multiples of what I had, I'd build a bunker, but I'd still much rather live on a planet where I can go outside, play golf, visit national parks, do woodworking in the yard etc.

The bunker is a backup to protect against outcomes you can't control, not the plan A to intentionally destroy the world.