Make sure you don't have children or anything selfish like that.
Should someone with the common cold be allowed participate normally in society? Probably yes, even though they may do slight harm others.
Should someone with Ebola be allowed to do the same? Probably not.
This is pragmatic line drawing. There is no alternative.
We let people generate a little pollution because we accept some is necessary for normal life. But beyond that should be regulated.
This belies a clear misunderstanding of the rights granted to government and its role
I want my government to protect me from my fellow man who sometimes wishes to harm me. If someone punches me or spews leaded petrol in my face, the government should intervene under the enforcement of the non-aggression principle.
Even minarchist conceptions of government understand this basic idea. It seems that yours doesn't.
Also the cause and effect isn't far removed. It's direct, just dilute. Person A is directly harming person B (and persons C, D, E, F, etc) by spewing leaded petrol into the air that person B uses.
There is no important difference between this and assaulting someone. One person is significantly harming another person (by giving them brain damage) without consent from that other person. That's the crux of it.
That said, I agree both harms need attention and the latter is more insidious.