Maybe you aren't good at setting boundaries, so people end up taking away your sense of self. I think it's something you could learn, are you sure running away is the best plan?
You grow to respect one another more over time when you share a foundation in things you are both equally passionate about. The amount I've learned from friends and also taught has made for very fulfilling life-long friendships.
The longer I've worked on things and gotten more into niche subjects the more difficult it is to casually relate to people, I've found that a lot of my 'friends' were just acquaintances. Knew them for years, but they didn't really know me and I didn't really know them.
If I do feel a need to unwind and casually hangout I prefer doing things like skiing, rock climbing, hiking, etc and keeping acquaintances in those circles. The reality is that if there's no shared foundation words become weightless.
That really resonates with me actually, when I hang out with old friends who don't share my work and the information I'm consuming. I feel they inhibit those parts of me, because my mind is trying to reconnect with whatever we have in common, which is by now a small and redundant part of my self.
Aligning your work and interests with your friends seems to be a powerful way to more happiness. It increases the amount of empathy you can have with each other, which is the fundamental way we connect.