I hope Sweden leaves too, and/or Brexit causes the union to collapse. We have paid billions in aid to the poorer EU countries for years and all we get back are refugees, Romanian beggars and criminal gangs from the Balkans.
In any case, neither the aid nor the 'consequence' you list above would be fixed by magic-ing the EU away.