I left a large Silicon Valley company for some of the reasons you mentioned and knew I shouldn't work anywhere like it again. It's just not a fit for me the way it is for others.

Smaller companies have a more diverse range of cultures. Maybe you got unlucky once, but it might not happen again. It's hard to know what the culture will really be like based on interviews, but learn what's most important to you and make a point of researching or asking about that as you interview. You are interviewing them as well.

And there are also small companies out there that aren't startups.