Hacker News Reader

AGI doesn't necessarily mean it will have initiative to think thoughts or do things on its own. An LLM, for example, "understands" language, but it can't, for instance, go out search Google for "how to escape a server if you're an AI" because it doesn't have the initiative to do that. A future AGI might have superpowers to build things we ask it to build, but may have no "thoughts" otherwise.

View on HN

It's true, not necessarily, but it's a real possibility, and I think that's kind of what we're aiming for, isn't it?

I'm pretty sure the richest AI builders (Facebook, Microsoft, Google, nation states and such) are actually aiming for AGI along the lines of the blogpost.

I don't think the rich and powerful will fund an AGI that has free thought and free initiative, but perhaps some other group will. The point of AGI, from what I can see, is to replace humans with reliable machines, so that they can do away with humans.

I guess I'm skeptical that a truly General AI could be constructed that didn't have those things. It seems like a fundamental contradiction to me. I don't think an intelligence could make complex good choices without understanding what the bad choices are, and to understand them I think they have to be possible.

Reply on news.ycombinator.com