That's a good question, because the numbers game is only imaginary if your populations survival isn't dependant on it. If we start diverting enough effort to rocketry, food prices may rise and the poorest can afford even less kcal per day...
But you are completely underlining my argument. I don't know if the rocket would get built.