Hacker News Reader

Comment by evandrofisico | Hacker News Reader

And self sustained nuclear fusion is 20 years away, perpetually. On which evidence can he affirm a timeline for AGI when we can barely define intelligence?

View on HN

And a program that can write, sound and paint like a human was 20 years away perpetually as well, until it wasn't.

Another way to put it is that it writes, sounds and paints as the Internet's most average user.

If you train it on a bunch of paintings whose quality ranges from a toddler's painting to Picasso's, it's not going to make one that's better than Picasso's, it's going to output something more comparable to the most average painting it was trained on. If you then adjust your training data to only include world's best paintings ever since we began to paint, the outcome is going to improve, but it'll just be another better-than-human-average painting. If you then leave it running 24/7, it'll churn out a bunch of better-than-human-average paintings, but there's still an easily-identifiable ceiling it won't go above.

An oracle that always returns the most average answer certainly has its use cases, but it's fundamentally opposed to the idea of superintelligence.

> Another way to put it is that it writes, sounds and paints as the Internet's most average user.

Yes, I agree, it's not high quality stuff it produces exactly, unless the person using it already is an expert and could produce high quality stuff without it too.

But there is no denying it that those things were regarded as "far-near future maybe" for a long time, until some people put the right pieces together.

This is the key insight I believe. It is inherently unpredictable. There are species that pass the mirror test with a far fewer equivalent number of parameters than large models are using already. Carmack has said something to the effect that about 10ksloc would glue the right existing achictectures together in the right way to make agi, but that it might take decades to stumble on that way, or someone might find it this afternoon.

> Carmack has said something to the effect that about 10ksloc would glue the right existing achictectures together in the right way to make agi

What does he know about that?

Well, he heads a company devoted to creating AGI, so admitting success in research is inherently unpredictable is surprisingly honest. As to whether his estimate that we have the pieces and just need to assemble them correctly is itself correct, I can only say it is as likely to be correct as any other researcher in the field. Which is to say its random.

Is this true? I think it’s equally easy to claim that these phenomena are attributable to aesthetic adaptability in humans, rather than the ability of a machine to act like a human. The machine still doesn’t possess intentionality.

This isn’t a bad thing, and I think LLMs are very impressive. But I do think we’d hesitate to call their behavior human-like if we weren’t predisposed to anthropomorphism.

> like a human

Humans have since adapted to identify content differences and assign lower economic value to content created by programs, i.e. the humans being "impersonated" and "fooled" are themselves evolving in response to imitation.

I'd argue we've had more progress towards fusion than AGI.

> I'd argue we've had more progress towards fusion than AGI.

way more pogress toward fusion than AGI. Uncontrolled runaway fusion reactions were perfected in the 50s (iirc) with the thermonuclear bombs. Controllable fusion reactions have been common for many years. A controllable, self-sustaining, and profitable fusion reaction is all that is left. The goalposts that mark when AGI has been reached haven't even been defined yet.

Yet at the same time "towards" does not equate to "nearing". Relative terms for relative statements. Until there's a light at the end of the tunnel, we don't know how far we've got.

Fusion used to be perpetually 30 years away. We’re making progress!

stop repeating that. first, it isn't true that intelligence is barely defined. https://arxiv.org/abs/0706.3639

second a definition is obviously not a prerequisite as evidenced by natural selection

> stop repeating that. first, it isn't true that intelligence is barely defined. https://arxiv.org/abs/0706.3639

I don't think he should stop, because I think he's right. We lack a definition of intelligence that doesn't do a lot of hand waving.

You linked to a paper with 18 collective definitions, 35 psychologist definitions, and 18 ai researcher definitions of intelligence. And the conclusion of the paper was that they came up with their own definition of intelligence. That is not a definition in my book.

> second a definition is obviously not a prerequisite as evidenced by natural selection

right, we just need a universe, several billions of years and sprinkle some evolution and we'll also get intelligence, maybe.

cool, you can move goalposts and claim no true scotsman would define intelligence that way; in addition you are confusing sufficient and necessary.

An Arxiv paper listing 70 different definitions of intelligence is not the evidence that you seem to think it is.

yes it is

Reply on news.ycombinator.com