Optimal for whom? Why should I as a pedestrian have to wait or take an alternate route (e.g. a cross-walk) just to save you a bit of time and energy? Why is your time more important than mine, just because you're driving a car? Why is a pedestrian's safety less important than a bit of energy? These are people we're talking about! If there are limits or inefficiencies about stopping and starting, that sounds like a design flaw in the car to me, not in our laws.
We have certain roads where drivers get priority -- they're called freeways. Everywhere else, you're a person using the road just like anyone else. If you can't keep track of a few people walking around, then you're going too fast! Slow down. Then you'll have more time to react, and it'll even take less energy to stop.
If I were to run on foot through the streets, tackling anyone who got in my way (after all, I would have wasted time and energy moving around them!), then I'd get charged with assault. They certainly wouldn't get charged for impeding me by getting in my way, would they? Of course not, I hit them! Why does anything change just because you're sitting in a car? Why does that give you special status, to the point where it's okay to endanger other people who might want to use the road?