I sort of agree, but to be fair a nice habitable planet could be a really rare and valuable resource in and of itself, assuming there are any life forms out there with biology similar enough to our own to find Earth desirable.
Also assumes there aren't a bunch of Earth-like worlds that are otherwise uninhabited that would be much easier to colonize than one with potentially hostile/poisonous natives.
or that the potential alien invaders are not adapted to a completely different environment than earth's