I think the biggest problem is not the brain drain, it's the uncertainty. I was planning to buy a house here, but I'm not going to now. This is bad for me and bad for the UK.
I know a few who have moved on too. Not many, but time will tell.
Since the Patriot Act and other changes in the early 2000s, your nationality is henceforth Sticky if you live in the US.
My sister expatriated last year (to UK, actually) from US, and she had to hire an immigration attorney, go through interrogations (from both destination and comefrom), and actually got sent back to the US once they found accidentally conflicting stories from her friends who currently live in the UK.
I'd never make fun of people who express desire to leave. Not everyone has the same circumstances as you.
I would have imagined that techies would appreciate this the most. The E.U is a bureaucratic, insatiable behemoth. Too many regulations and I would like to think that this is something techies have a great disdain for. I mean, just look at how unfairly the E.U has treated co.s like Google, Microsoft and Uber - these companies have deliberately been attacked by the E.U using antitrust laws among other regulations.
How has this helped the situation? I'd say, in no way. Stopping American companies from thriving in the E.U won't help the situation. Instead, learning from previous mistakes and correctly anticipating/predicting the next wave would be greatly rewarded by market forces and this would be the best thing for individual, sovereign European countries.
Please watch this video about the insane E.U laws I've mentioned: https://www.youtube.com/watch?v=44YTTyQKyJQ