I used to dismiss the idea of an impending singularity. Now I'm not so sure.
Hopefully AGIs will reach hard physical limits to self-improvement, before taking over the world.
* Birthing prematurely compared to other species (most species live young are functional at birth) in order to fit the brain through the hips. * Permanently reducing the mothers top speed (wider hips means the joints are exposed to higher forces while walking). * Requiring 5% or so more food.
All three of those have been serious impediments to human survival for most of our existence, and provided a counter-pressure against improvements in biological intelligence.
In the past century, the Haber-Bosch process has meant we can reliably grow enough food without pursuit hunting. Without that counterpressure, evolutionary improvement seems very likely, but will take hundreds of thousands of years without genetic engineering or eugenics (neither of which seems like a good idea to me).
[0] https://journals.plos.org/plosone/article?id=10.1371/journal...
However, women need hip replacements at a far higher rate than men. Given men are typically heavier one would expect their joints to wear out faster, but as alluded in my comment upthread, widening the pelvis means walking exerts a higher peak force on the hip joint due to increased torque, which is a plausible mechanism for it wearing out faster.
Somewhere close to our current level of intelligence, the benefits of increased intelligence stopped justifying further improvement. The question is why here?
Birthing issues seems like a particularly weak explanation, given than that the negatives of bigger hips scale with the radius of the head while the benefits of intelligence scale with the volume - the more intelligent we get, the more it makes sense to keep making the skull larger. With increasing skull size, there is no need for babies to become any less functional. Calorie demands should scale linearly with intelligence, so it's possible increased intelligence doesn't help a hunter gatherer acquire more food beyond a certain point, though given how much food production has been increased through ingenuity this seems unlikely to be the case.
Implying linearity here is ridiculous. The increase in force applied to the hip joint scales with the radius of the head; cartilage gets damaged if you apply too much force but will last a lifetime if you don't.
> the benefits of intelligence scale with the volume
That's completely unsupported. Firstly because the number of brain cells scales with the volume, but there's no evidence that larger brains in humans are associated with higher intelligence (then again, most mutations are negative), and secondly because it implies a linear relationship between 'amount of intelligence' and 'amount of benefit'.
> given how much food production has been increased through ingenuity this seems unlikely to be the case
This has not happened on evolution-relevant timescales. There was famine in eastern Europe only 50 years ago. People died of hunger because their calorific needs were not met - selection pressure towards surviving famine has not stopped.
Another possibility is that other species are constrained by us, and we have manipulated our environment to reduce the survival advantage of increased intelligence for humans as a whole.
That is total speculation, and also it's not necessary to assume so much to have a similar outcome. It seems much easier to imagine an AI that is 2 or 3 times smarter than a human, and has very fast transfer of knowledge with other compatible beings.
I think that's enough for them to take over, if there are enough of them.
But anyway it seems obvious to me that we absolutely should avoid trying to build fully autonomous digital creatures that compete with us. We should rather aim carefully for something more like a Star Trek computer, without any real autonomy or necessarily fully generalized skills or cognition or animal-like characteristics/drives.