Try to find a single reliable source supporting this claim. You might be surprised to find that it doesn't exist, and that it's also an example of citogenesis. [1] This is another perfect example of what I'm talking about. After Muslim armies invaded the Iberian Peninsula they created a system of government with a tiny minority of Arabs at the top with everybody else treated as distant second class citizens. They started trying to force people to convert and imposed taxes and other penalties on those who did not.
The predictable rebellions against this were the start of the Reconquista. It spanned many hundreds of years but was almost all extremely small scale. And they weren't driving anybody out in large numbers. The Arab and Berber tribes never engaged in mass migration or anything like that. Iberia remained overwhelmingly native Iberian with a tiny Arab elite. The same Spaniards and Portuguese you know of today are the ones that were there under Islamic rule as well.
[1] - https://xkcd.com/978/