In the Pali Canon, the Buddha is often referred to as “the recluse Gautama”. Seems like he found a way to have a pretty good life though. Your generalization doesn’t account for the other path in history where hermits got away from the maddening crowd and found peace of mind and knowledge.
You’d prefer to live in society, and that’s fine, but everyone isn’t wired the same way. They may be outliers, but I would disagree with saying that they’re “doomed”.