This is one insight about actual geniuses that shun the "classic" trappings of life that has always intrigued me. They are essentially coerced by their own neurology to aim for academic pursuits, because anything else cannot scratch that itch. Money and power are nice (and Johnny von Neumann loved his parties and loud music) but in the end it's just not that stimulating in comparison to hard abstract problems. Social relationships still have their draw as well, but you have to hit the knowledge-pipe regularly or face withdrawal.
For people of that caliber, living in pre-modern societies must have been equivalent to torture. For the rest of us, it would have been like watching paint dry for our entire lives. Sounds like sour grapes, but in a way I am grateful not to have to deal with all of this. Looking at von Neumann's example again, the fear of death still caused him to lash out irrationally and in the end even he could not escape the human condition.