The objective truth is that globalization and labor arbitrage overall produce value. That is why we are seeing markets around the world react negatively to the UK's decision. Global economies will work less efficiently in a world in which the UK is not part of the EU.
However as a society we are doing a bad job of distributing the increased value that comes from globalization. Int is true that the way we are distributing the value is leaving the people you are talking about worse even off than before. That is the problem and it will only get worse as other countries continue to develop and more jobs are taken over by automation. The way to solve the problem is not to roll back progress and retreat back into isolationism. That would leave society as a whole worse off. The answer is to continue to encourage progress but increase redistribution efforts to help the people who are left behind.
It is simple math really. Imagine a scenario before globalization where person X has $10 and person Y has $30. Now after globalization person X has $5 and person Y has $40. Person X would clearly be angry about this development but that doesn't mean the solution is to revert everything to how it was before globalization. The answer is to take somewhere between $5 and $10 from person Y and give it to person X. That way everyone is better off than before and we all benefit from progress.