I know some homeowners hate dandelions with a passion. I don't, and I let them grow. But there's no way my dandelions aren't invading my neighbors' yards (fortunately they don't give a fig). And if they sprayed pesticides to get rid of those dandelions, no way those pesticides aren't blowin into my yard. So both of us would be unhappy.
I had a family friend who inherited an old olive orchard in California. He let it go wild, and it became a little wild-life park. It was fantastic. But the neighbor farmers didn't like it, because they believed that it harbored pests that would eat up their crops, and weeds that would compete with their crops. So the farmers complained, and the county forced him to mow it all down.
Of course, all of this is about which externalities matter, and when, and to whom they matter. In a fair society, that requires examining the public purpose, what priorities we set as a people. In an unfair society, it means who has the power to get their way.