The sea people sounds alot like "ship"-residing refugees sailing along the coasts to me. They would be armed, they would be desperate and pillage like the vikings. And they would end wherever there boats ended up falling apart. Either fighting the locals or mixing with them. I guess this is what you do, when your city is besieged, runs out of food and you still have an open harbor.