Top Reasons AI Will Never Become Superintelligence

Even if this were valid, which it isn't at all, I keep returning to an apparently Japanese WW2 quote, roughly "I can't help but notice how all our great victories are getting closer to home".

1. Nobody claims AI is "alive", this is an irrelevant argument.

Following the link, then getting #5:

  If the flipped paper alive? Intelligent? Does it have a will? Has it reached true general intelligence? Is this a mind? Is it conscious? If so, where is it? In the flipping? In the paper? The ink? Just where?

Chinese room. Or: Show me one atom of consciousness in a human. Where is it?

2. "Computer models do not learn: they are incapable of learning" - computers change state from being unable to do a thing to being able to do that thing via a process of observation and practice. If this doesn't count as "learning", one can make the same statement about humans.

(I would however say that all machine learning processes are very slow learners; but this is a different statement to "incapable of learning").

Gödel Test: "The name is still good, because in essence, or if you like, Gödel proved new axioms" - this is a fundamental misunderstanding of the words "proved" and "axioms".

3. Is exactly what it accuses others of, simply choosing words to suit the argument

4. Unclear what the point is, starts with mode collapse in AI then becomes humans also doing this (so how is this "AI can't be like humans"?) then it's argument-by-sigmoid (yeah and?)

5. "And so we are on our way to pantheism. Or to realizing we cannot escape the we in this." - that's not "pantheism", it's panpsychism. Pantheism is many gods.

"It is we who make our machines, and thus we who command their functions." - "thus" does not follow, as demonstrated by every malfunction in history.

Even if you chose language such as to make it follow, this is not an argument for safety: we built Chernobyl, we looked at the disaster and understood it with hindsight, but we did not foresee it.

6. "Our minds are instead in continuous space" - false.

"Some of our thought, that which makes us rational beings, is itself not material, and does not happen in matter (see also: entanglement)." - zero evidence for non-material processes involved in our thought processes, but also if this "entanglement" is supposed to be quantum mechanical entanglement this is another fundamental misunderstanding of these words.

No basis for claim "No physical process, which are mere arrangements of matter, gives meaning to objective facts;", and by extension conclusion is moot.

7. "There is no algorithm that can capture all of Reality: it is a false hope to believe that some secret algorithm will be discovered in imperfectly and incompletely measured Reality to predict all Reality" irrelevant argument, nobody claims this, and humans also cannot.

"In particular this means the jobs of mathematicians are in no danger." - at this point, with Millennium prize puzzles falling, the word "lol" comes to mind.

8. "They are also right to dismiss arguments which say the “soul” is some kind of external thing that imbues an organism with life." - amused by this, contradicts prior claims of non-physical components to human thought.

If "Substance is more than the sum of parts", then one can easily argue the same for emergent behaviours in machines.

9. "The Turing Test Is Trivial" - fundamentally WTF, this was a "holy grail" since before the author was born.

(Of ELIZA) "It convinced many it had passed the Turing Test, and so by that measure did pass the Turing Test." deeply misunderstands the Turing Test.

"Many also believe many machines are possessed of a spirit, like pachinko machines. Yet these follow only simple physical constraints, even if they are of sufficient complexity to be unpredictable by most." - as does the human body. You can't just will yourself to fly, or to be immune to fire, or to shape-shift, we're all subject to simple physical constraints.

10. "Unpredictability is not a requirement for life." more irony: while everyone working on such models would agree, the unpredictable nature of reality is used earlier in this list, by the author, to argue that what machines do isn't thinking and therefore they're not alive.

No, what unpredictability brings is risks.

"If, per impossible, machines were to come alive, they can be switched off. Simple as that." First: yes, and? Humans can be shot, that doesn't make us not a threat to each other.

Second: They're getting good at hacking, and downloadable models can run on consumer hardware, which means they can do a neat trick which humans only approximate by writing books - copying their processes identically to somewhere else, surviving the off-switch on the original device.