It's in the name isn't it? Colonies. There were already people there who shouldn't have any rights to the land.
For large swaths of the Americas, these are all different groups, many of which seized the land through violence.
It's something people like to throw out as an argument, but it falls on its face under the most basic of scrutiny. The entire earth was "stolen" many, many times by this definition, making it nonsensical.
https://www.history.com/news/native-american-broken-treaties