What you say is true but most objects are not starting from rest at infinity. Things thrown up from the Earth, for example.
It also neglects that there is an atmosphere...
It also neglects that there is an atmosphere...
I think it's implied that the impacting objects come from outside the Earth-Moon System. Otherwise there would obviously be no minimum impact velocity.