Loss of power, prestige, and influence. German not UK law and citizens will safeguard that tremendous amount of assets.

with the current Tory government the chances of taxing those assets significantly is indeed low.

with the assets out of the country the chances of taxing them at all is zero. As well as losing any possible indirect benefits from them being around (not saying there are benefits, but we certainly won't get them if there are)