I think that it might be overblown or not affecting technology departments.
I would have also thought that if 20% of the workforce were relocated it would be visible in the canteen, etc.
Not saying that a relocation isn't happening at all, just saying that it's not visible within a bank.
My last company opened a new office in non major US cities last year. The next day the newpapers published the article <Corp is moving to new town after Brexit>. There is no relation, we had to get back to them to redo the announcement.
About my current bank, there is a published article from a major media outlet specifically saying that my department stopped hiring and is being moved overseas because of Brexit. It couldn't be further from the truth, we've been hiring dozens of people over the last years. I don't know who to complain to to have this removed.