Comment by Melatonic | Hacker News Reader

Comment by Melatonic | Hacker News Reader

Parent Full thread

I think it's bigger than that. There's an unsaid anxiety around this that I don't think people are properly quantifying yet but is probably bubbling around inside their subconscious: we may soon get to a point where computers and machine intelligence outstrips our own and is able to solve mathematical and logical problems that are not just unsolved because they are hard but because we are hitting an inherent limit in the concepts and problems our physical brains can comprehend.

Demis Hassabis has talked about this a bit in regards to theoretical physics and it's an idea that I think I agree on. Every once in awhile we randomly get a super genius who's brain is seemingly able to conceptualise or visualise beyond the state of the art. But at some point I think we have to accept that we are just animals and inherently limited by our brains (and means of observing the physical world).

The scary part isn't that AI is solving the unsolved problems that will force people to act like students again. The scary part is that we may not be able to understand the work needed to get to the solution at all.

Obviously we're not quite to that level yet but it's entirely possible we do get there. And on the optimistic side ~ maybe that's what we need: better tools.

This reads to me like Victorian fears of buildings over three stories (you'll asphyxiate) or the idea that humans will die instantly when moving faster than a horse could carry you.

I see no reason whatsoever to expect that humans have any kind of limit to what we can understand as a species. We've had more unthinkable and incomprehensible discoveries than anyone cares to count. They become understood, normalized, and folded into our scientific canon. We went from inventing airplanes to landing on the moon in a matter of decades. We went from banging rocks together to quantum physics and string theory. I think we'll be alright.

Hypothetically, if a solution to a problem requires a billion intricate steps there is no way for humans to comprehend it. Human life span is too short. We can't even grasp what it entails. If a machine can pump out humongous proofs and solutions on a regular basis we will never catch up.

I'll add to it. The classic answer to your objection is "but as a species we'd have sooner or latter 1 billion steps worth of intellectual capital". Today someone gets to an intermediate result worth 1 million steps, tomorrow another million and so on.

But there's no guarantee that such "low IQ" intermediate steps exist. We do well with many classes of problems because of divide-and-conquer. But not all problems can be solved through divide-and-conquer.

For a rough analogy, see how AI plays chess and intermediate board positions often don't make sense to us. Imagine now an enormous chess-like game played in an enormous chess board where a match lasts longer than a lifetime and none of the intermediate steps make sense to any human.

Reply on news.ycombinator.com