To pick an extreme example, should you steal something that's lying out in the open in a dodgy area because someone else will if you don't? Some things you do for yourself, and an action isn't required to have an effect for it to be moral.