I think this is quite right. But this is even true if the racism is unfounded. The mere illusion that some ethnic groups are low-contributing will weaken the cohesion of society. So in this backwards way, racism becomes a self-fulfilling prophecy.
It's for that reason that the identity politics of today's social justice advocates infuriates me. These activists can't see that their divisiveness is actually bringing them further from their goals.