Only because pollution alerts them to your presence. Walk near enough to them and they'll attack you, pollution or not.
There's also the expansion groups, which will periodically move around to create new nests. Even if you don't walk at all, you will be found and attacked sooner or later.