I'd wager that the OP hit the nail on the head here. There are many large Pakistani communities in England in areas such as the Midlands, East London and Bradford that do not inter-marry, socialise or integrate outside their own culture, and when after two or three generations this remains unchanged the perception to the locals is that the cohesion of their communities have taken a real turn for the worse.
The EU's inability to manage middle eastern migrants in a controlled way, along with the belief that most will eventually hold EU passports and speak rudimentary English I think were major deciding factors in those communities, communities that have had demographic shifts not seen in Scotland and Northern Ireland & whose concerns are shut out and branded racist and xenophobic by both the media and the left.