Sorry, what about the brain drain when those "European expats" came from their countries into the UK?
Or that doesn't matter because it was all "within EU" anyway?
Still, UK hadn't paid for their education and other benefits while they were growing up in their native countries, and their native countries didn't get taxes from those people while they were living in the UK.
And those countries would also like to have their scientists etc stay and work there to improve the country's industry and economy.
Brain drain works both ways. You can't lament brain drain from an economy that actually caused brain drain, and now those brains leave it.