In the late 70's my Uncle had a run of bad luck and a dubious business partner basically sink him financially. After some discussion, he made a plan. It was simple:
Uncle goes back to Virginia to lay low in one of the back hollers.
My mom gets his mail, due to him living with us for a while[1], and writes "Deceased" on it, and "Return to Sender"
5 years pass. Maybe 4, I can't remember.
Uncle shows up, everything is fine, and he and my aunt live out the rest of their days in a small comfy trailer no worries.
[1] Living and other time with this Uncle was a great time in my life.
Edit due to device change: My uncle had one eye, the other lost when he was young and unlucky in the woods. He read everything and acquired a great many skills which he proceeded to pass along to me: lock picking, electronics, engine rebuilding and a ton about autos, working with wood, metals, tools... He is probably still doing that in his afterlife. Good soul who I treasure having known.
So, he took advantage of that by essentially living out of range. Once all the companies charged him off and basically moved on, he was free to come back and pick up where he left off with few worries.
It turns out that in a world of "people forced to leave the village to live a new life free from the consequences of their prior deeds", the main reason people would try to start living in a new village was because they had done something that made them no longer welcome with all the people they knew before.
One oft-understated advantage of an explicit noble class is that it provides a medium for verifying "this person really is traveling for legitimate reasons".