If you don’t want to talk to them, why not just say so? “I’m afraid I’m busy, I hope you understand” or “Excuse me, but I would prefer to mind my own business”. They might not love the response, but no sane person would continue the conversation after that.
Also, out of sheer curiosity, may I ask why you would dislike the interaction?