The difference I see is that I'm not personally at risk from someone chomping down on fast food. But I may well be at risk from someone driving unsafely.

A lot of countries ban smoking inside non-residential buildings e.g pubs out of concern for non smokers.