Comment by rhacker | Hacker News Reader

Comment by rhacker | Hacker News Reader

Parent Full thread

I kinda see it this way:

AGI is Data in Star Trek TNG - trying to be human, making decisions to want to be alive, eventually dreaming and finally using an emotion chip. Another alternative here would be Moriarty or the various doctors in Voyager.

AI is the Ship in TNG - lots of heuristics to figure out what the user is trying to do. Past usage of commands and relating major events outside the ship with algorithms for battle, life support, etc.. Events categorized by importance and automatic handling to save lives when necessary. Basically an extremely advanced Siri that doesn't really misunderstand you - while at the same time not really caring about you or knowing anything about being alive other than the priorities built into its software.

I think for the next 100 years we're going to have AI progressing like the ship in TNG. I don't think we'll have AGI until maybe 100-200 years.

Then again when I was born no one had a fucking clue eventually we would have something like the iPhone and talk to someone in China with <1 sec lag. So my estimates could easily drop to half.

Your comment reminds me of this xkcd cartoon: https://xkcd.com/1425/

It's 5 years old now (coincidentally the time span quoted to develop a solution), but recognizing a bird was already considered a solved problem 3 years ago, less than 2 years after the publication of the cartoon.

Predicting the future is hard.

Surprisingly, recognizing a bird is much harder when you count rare species and running birds. Thus, sparse data. Does it recognize penguins too?

AIs today still fail at it. Some folks were trying to train one to match endangered species and they had to pull mighty tricks to have some 70% accuracy. I think it was here on HN some time ago, but can't recall a link.

And yet, average humans can classify even fewer species.

Also after extensively studying ornithology?

With the same amount of training/data as NNs? I doubt it...

Do you take into account billions of years of training due to evolution?

Technically that’s network architecture, not training data... admittedly though humans are “pre-trained” from birth.

AGI as discussed by people working on it is closer to the Ship than to Data. The defining thing about AGI is that can figure its way around arbitrary challenges just like humans do, but not necessarily the same way humans do. There's a concept called "orthogonality thesis" that tells you that intelligence and values are orthogonal. That is, there's no reason why a powerful enough AI would have to develop values similar to those of humans (like Data, trying to be humans) - you could have an AGI that's smarter than humans in everything, but is "not really caring about you or knowing anything about being alive other than the priorities built into its software".

> not really caring about you or knowing anything about being alive other than the priorities built into its software".

Is that really "general" intelligence, then? I would argue we don't know today whether consciousness and agency are separable from general intelligence. You could say that "general intelligence" implies some degree of intelligence on any topic, which would include self-reflection and metacognition.

It sounds to me like you're describing something like a p-zombie[0], which we don't know to be able to exist.

[0] https://en.wikipedia.org/wiki/Philosophical_zombie

I believe it is. I'm also not describing a p-zombie, since I find the whole concept of p-zombies utter nonsense.

The general AI I described would have self-reflection, agency and arguably consciousness, yet - per orthogonality thesis - it may not have anything resembling human values.

So like, a sociopath, then? I guess we do know that those can exist.

Yeah, sure. The orthogonality thesis essentially implies than a GAI randomly plucked out of space of possible minds will likely be considered sociopathic by our standards. That is, if we can comprehend its thinking at all. "Not sociopath" is a very particular set of values.

> a GAI randomly plucked out of space of possible minds

Sorry, but I don't think you have any rational basis for imagining what the "space of possible minds" represents. The only minds with human-level intelligence we know of are human minds that have (with variation) human values.

The claim you're making is analogous to saying "any extraterrestrial life we find won't be carbon-based because out of the space of all possible substrates for life, carbon is a very particular one", but that's an ill-founded supposition because we have a sample of N=1 and maybe carbon-based life is the only kind of life there is.

Maybe a true GAI mind will be "like us", maybe it won't be, but we don't have anywhere near enough data to speak with confidence about it.

Reply on news.ycombinator.com