And I would note that Neumann, despite being happily married, a gifted negotiator, lady's man, and party-goer, never had children.
So Neumann might violate either the simple or reproductive fitness requirements: to the extent we can hope to manufacture Neumanns through genetic engineering, it's not going to be easy; and to the extent we can judge from n=1, the reproductive fitness penalties may be large.
I think the source of the problem is rather our very limited understanding of maintaining and improving the brain functions. As far as my limited knowledge goes, we still cannot reverse many dementia-inducing illnesses; once we see deeper into them, maybe better supplementation will appear. It's understandable that various "trivial" methods at increasing IQ fail.
(Heck, we don't know much about benefit/hazard of many chemicals that enter our body; and even when we do -- smoking, obesity -- people tend to disregard the advice.)
I know it because I have trained a bit for some IQ tests and the results improve the more you train for it. That's something everyone should be capable of.