If you're not allergic, you're not allergic. Then they're just there, eating your dead skin cells. You shouldn't worry too much about microorganisms unless they cause you problems -- they are everywhere, and you can't get rid of them.
On the other hand, dirty water has killed a lot more people than allergies have.