I 100% agree with you but I would say that many places will turn the lack of participation in Friday beers, book club, and gokarting back at you as not being a good culture fit.
I'm from Iowa where the work culture is much different - even for SWEs. I prefer it to this weird "culture fit" environment out here on the coast because I am paid to work, I am paid to be a software professional. I am most happy when I am working, and I am most happy when I am effectively making software that solves people's problems. If I want to stick around and focus on my assigned work I feel that should never turn around and bite me in the ass... It does consistently.
I've made a huge shift in my career where I realize that I have to go against my own wishes and participate in the gokarting, beers, game nights, etc. You have to go - every time. You have to make as many friends there - every time. My career has gotten much easier the more social I've become. I just resent it because I want to do good work, and go home. I have a family god dammit.