For one, I've seen plenty of non-motorized bikes go over 20mph in bikes lanes, particularly down hills.
> zooming by me and my infant in a stroller at speeds that are too dangerous to be on paths
To be clear: you and your infant are not sharing the path with this bike, right? It's on a bike path, you're on a pedestrian path. I don't understand what a "pedestrian bike path" is. If a car drives past the sidewalk at 25mph do you have a similar reaction? If not, why not?
And just so we don't get too many "won't somebody please think of the children!" accusations thrown around: I own a cargo e-bike and I take my kids to school on it every day, via a separated bike path. If I were required to bike on the road instead I simply wouldn't do it, it's far too dangerous. So we'd probably end up having to get a car and if, heaven forbid, I were to get in any kind of an accident a pedestrian would be considerably worse off.
EDIT: From the replies I can see many cities do have shared paths so I take that back. Seems like a very unwise choice and goes back to the original point: 90+% of of road space being dedicated to cars is the core problem, not bikers. And plenty of non-motorized bikers travel at unsafe speeds so I maintain a blanket ban on e-bikes feels like it isn't coming from anywhere logical.