You grow to respect one another more over time when you share a foundation in things you are both equally passionate about. The amount I've learned from friends and also taught has made for very fulfilling life-long friendships.
The longer I've worked on things and gotten more into niche subjects the more difficult it is to casually relate to people, I've found that a lot of my 'friends' were just acquaintances. Knew them for years, but they didn't really know me and I didn't really know them.
If I do feel a need to unwind and casually hangout I prefer doing things like skiing, rock climbing, hiking, etc and keeping acquaintances in those circles. The reality is that if there's no shared foundation words become weightless.