Your problem is that you, as you say, equate smoking with poison. That's hyperbolic. One could say the same about sugar, but it would sound just as ridiculous.
It's almost as if smokers are dehumanized these days, and non smokers deliberately ignore reasons to smoke.
There has also been ample evidence of neuroprotective effects of nicotine[0][1] for decades now. I dont mean to suggest that you go out and start a pack a day habit, but my point is that walking past a smoker won't give you cancer.