If you're robbed by bandits you can choose not to travel on that road again. That doesn't mean that you weren't robbed the first time.