How about this solution: If your face itches or something, use the back of your hand to rub it? Or some other part of your body that you can be confident isn't used to touch anything potentially contaminated.
Regarding washing hands, I've found that if I do that too much then my skin (starting on the back of my hands) tends to dry out and flake off. Certain kinds of moisturizing soap are less bad at this, but I don't choose what soap is available except at home, and it has led me to find other solutions like the above.
Edit: this may be a potential vector for reinforcement learning