Hacker News Reader

Comment by version_five | Hacker News Reader

>GPT can never be an intelligent agent because it’s not an agent

For all the handwaving people like to do, this is the bottom line, and it's not clear to me why researchers haven't spent more time clearly formulating this, vs a lot of proving that various tasks and games don't actually require intelligence.

As best I can formulate it, "AI" can't "escape" its programming, and just a neural network even less so. There is no mental model of the world to rationalize against, and if there was, it would be pre-defined or coded in some way that it still couldn't be escaped. Whatever tasks a ML model has been deliberately programmed with just have nothing in common with intelligence. Scaling doesnt change that.

I subscribe to the idea that with ML we may have understood how some of the sensory mechanisms work, like visual recognition, but we're not actually closer to the intelligence part.

View on HN

The same author has a story describing how AI might escape: https://www.gwern.net/fiction/Clippy

Not that I agree with the author but 'AIs are sandboxed and resourced constrained therefore safe' doesn't feel very convincing to me after reading that story.

> "AI" can't "escape" its programming

I don't think you can back that claim up. ML models as often used can't escape their programming. But I don't see any fundamental reason you can't loop one back on itself.

Isn't this already done to some extent by using a model to assess its own output versus the current environmental state? Recurrent models combined with reinforcement learning actually look a lot like what you're claiming can't be done.

> There is no mental model of the world to rationalize against

There's a theory and some associated research about a model that performs well necessitating an internal model that accurately reflects the environment in which it operates. Or something like that. Sorry I can't recall the paper off the top of my head.

> Whatever tasks a ML model has been deliberately programmed with

Yeah, no, that's not the case. See the entire field of reinforcement learning.

> Isn't this already done to some extent by using a model to assess its own output versus the current environmental state? Recurrent models combined with reinforcement learning actually look a lot like what you're claiming can't be done.

Yes, you can do this. And I believe you could recreate human intelligence if you made something exactly like a human and put it in an environment exactly like a human - but for some reason the AGI theorists think AGI via AI research is scary and "N"GI via having children isn't.

But GPT doesn't have those features, which is why it's not one of them.

> Yeah, no, that's not the case. See the entire field of reinforcement learning.

ML models (especially big ones) fluidly combine learning with memorizing, which is cool, but it's more like "compression" than "programming".

Reply on news.ycombinator.com