They also met for lunch when he was a well-connected financier. He didn't exactly go to his island
Which is why these people were meeting with him like he was a well-connected financier.
The people who were meeting with him weren't checking his bank account balances to see if he was really a billionaire or how he got his money.