To nitpick a bit, it is true that the richest apartments in London and Sydney are often owned by people who live outside England or Australia. But that is not evidence that "the richest Britons" live outside England; it can easily be the case that the richest Britons are happy to stay in Britain and the richest apartments are owned by a class of foreigners richer than the richest Britons.
Whereas, if all the richest Swedes live outside Sweden, that does tend to suggest that something made them move. (And doesn't suggest that the apartments they maintain in absentia are the richest apartments.) The default for Swedes is to live in Sweden.