It was more about local residents becoming Arabs than Arabs migrating in large numbers. Palestinian Arabs are mostly descended from ancient Canaanites. More generally, the demographics around the Eastern Mediterranean have changed surprisingly little since the Bronze Age. Most conquerors just replaced or took over the old elites. Many people died in the process, and the survivors would often adopt a new culture, language, and/or religion.