Hacker News Reader

Comment by boyadjian | Hacker News Reader

Maybe it is a good thing, so the singularity will be avoided. There is so much wrong usages of AI.

Mathematician here, I don't think the singularity as popularised through people like Ray Kurzweil would happen.

In particular, their metaphor of an "infinity" point seems to me like the incorrect application of mathematical ideas to social contexts.

I think the question of artificial general intelligence is a different question than a singularity, much like how the Chinese room addresses a different question than the Turing test.

Why is it a different question? AFAIK the assumption is that if the computer can improve itself (which it can if it has IQ 100), there is nothing holding the singularity back.

Which is dead wrong, because not only is there no usable definition of "improve itself", but there isn't even any understanding of the kinds of skills required to create a usable definition.

It's the difference between a computer that is taught how to compose okay-ish music, and a computer that learns spontaneously how to compose really really great music and do all of the social, cultural, and financial things required to create a career for itself as a notable composer and then does something entirely new and surprising given that starting point.

They're completely different problem classes, operating on completely different levels of sophistication and insight.

A lot of "real" AI problems are cultural, social, psychological, and semantic, and are going to need entirely new forms of meta-computation.

You're not going to get there with any current form of ML, no matter how fast it runs, because no current form of ML can represent the problems that need to be solved to operate in those domains - never mind spontaneously generate effective solutions for those problems.

> Which is dead wrong, because not only is there no usable definition of "improve itself", but there isn't even any understanding of the kinds of skills required to create a usable definition.

I disagree. A program improves itself when it reacts to a problem and implements a solution. Obviously that is very general, but enough. A human of IQ 100 certainly can develop software; a program of IQ 100 should be able to do the same, and then you scale horizontally.

Have you taken an IQ test? They only test a few classes of problem. Performance on these for a human is deemed a workable poxy for intelligence but, for something that approaches the problems very differently, it may not be at all indicative of general intelligence. I think we probably already have reached or are near the point where we could train systems to achieve human-level performance on each of those basic tasks. We are, however, not seemingly near a humanlike AGI.

I don't mean to be rude, but I'm impressed that you seem to have completely skated over the content of my comment.

Please read it again. You need to understand what a "problem" and a "solution" both are - in detail - because otherwise you have nothing to work with.

And a human of IQ 100 will only ever develop poor software. If you scale horizontally, you won't get game-changers - you'll just get a flood of equally poor software more quickly.

Many here have an IQ well over 100 and have no idea how to rebuild an improved version of themselves or even how their consciousness works internally. Seems rather likely any sapient AI we cook up will have just as little idea how itself works as we do how we work. QED, no singularity.

A human with IQ 100 can develop software. Can they develop it well enough to improve AI software? Or can they just adequately develop general software?

Can a human with IQ 100 write general AI software that a human with IQ 100 can debug?

There are different ways of explaining this (like conversely talking about what computers and computation are) but my personal preference is to explain it in terms of time.

We don't understand physics well enough to know whether time (and space [1]) can be arbitrarily halved over and over.

For an actual "infinity point" you would need such halving of time of computation up to a point where you actually have infinitely small time increments. The way you would define this would be via a limit and again the main premise is arbitrary halving, something that we have not seen in nature and which is a mathematical tool, rather than a hard physical fact.

Since you have physical computers, the composition of such a computer would have to lead to computations in these infinitesimal time brackets and in turn, for example, may necessitate infinitesimal matter. The way I understand the Planck length, at least as a concept, is that after some really small length of space then essentially you don't get anything smaller.

In any case, talking about a "singularity point" is an activity done by social personalities who like to sound clever. But they are not saying anything remotely clear.

As I mentioned, the other argument that one can make is to challenge the ability to indefinitely "improve yourself" as a machine, another dubious claim.

[1] https://en.wikipedia.org/wiki/Planck_length

Edit: I am not a physicist, so there could be better ways to think of Planck length and discuss its validity.

See my comment below about whether IQ 100 is enough. But also, there still can be plenty holding the singularity back.

Remember Zeno's paradox. "Improving" could mean "incrementally approaching a limit". It doesn't necessarily mean "marching off to infinity".

Then, there's the matter of hardware. An infinitely smart AI running on finite hardware seems likely to be a contradiction. Is the AI going to be able to not just improve its code, but improve its hardware? Without depending on slow, ignorant humans, who might even be having second thoughts about the wisdom of giving this AI even more hardware? That takes considerably more than IQ 100; that takes robots carrying out the AI's wishes, which some factory needs to make. And an IQ of 100 probably isn't going to cut it to hack all the factory control systems, to take them over and have them start producing what the AI wants.

Reply on news.ycombinator.com