As child mortality drops, so do birth rates. The effect just lags a bit as people change their behaviour. Healthy secure people will have less babies, because they don't need to have so many.

Letting a small number of people die of hunger is not going to change that equation. In fact it may have the opposite effect.

And food aid is a small minority of all aid. Long term programs to build local capability are already the vast majority of money spent.