Only sort of true. If you naively apply the rocket equation like you're imagining, you would predict a nonzero final velocity for a hypothetical rocket that has enormous ISP but thrust less than its weight. This is true if the rocket is starting out in orbit, but it's totally wrong when you're on the ground. Starting from the ground, you also have to fight gravity, so you care more about (thrust - weight) / mass.