How old were you when you did this? It sound like you were beyond the typical early 20's backpacker age - weren't you the weird old guy around? (I feel like this in many situations already, I'm 31).
I was 27. I quit after 3 years out of grad school. But you are right. I dont think its just age. It is also the differences in the backgrounds. You just need to adapt and mirror the other person. It is easier said than done but my social skills improved a lot because of that.
If you're relatively socially aware you should be able to tell the difference between being the weird older guy who is nonetheless awesome to have around, and the creepy older guy who can't tell when someone wants him to leave them alone.