The way I understand what I've found is, the islands were not inhabited before the Europeans arrived there, and "Chagossians" weren't native. They were brought in as slave workers after the settlement. (Then, the population expanded with voluntary migration)