While on paper you are correct about the African migrants in reality people were rightly concerned that the will to enforce this was weak. The issues in the English Channel still show this problem exists. And Europe continues to show little desire to actually take action in this problem and so we are seeing similar sentiments arise in Italy as lead to Brexit.
Imho the real blame for Brexit lies at the feet of Angela Merkel. Brexit would never have happened if she hadn't opened the borders in the Syrian refugee crisis. While this was a very open-hearted and kind gesture, as a geopolitical action it was disastrous.