Should we ban cars too? Babies die in there as well. Also alcohol. And music/movies - those can distract from a baby.
What's missing from your story is any sense of scale. For pretty much any human activity I can find you a dead baby, including over-zealous baby care. The question is how common is that compared to other activities, and weighted for the benefit of said activity.