Hacker News Reader

Comment by sph87 | Hacker News Reader

> AGI is already here

I feel like there has been a ton of noise about this, but frankly, no one has actually defined what AGI means. I feel like the goal post is constantly shifting.

Take for example Humanity's Last Exam. It is so broad and complex that while an individual in a specific field might be able to answer their specific area of questions, they certainly would not be able to achieve >50% on the total question set.

There is this idea that AI has to be perfect to be intelligent - but we consider Humans intelligent and they are not even close. So is it the ability to generate novel ideas? Prove theorems? Pass tests?

I am not arguing that rote memorization is intelligence, or that we have achieved it, but does anyone know what AGI actually.. is?

View on HN

> I am not arguing that rote memorization is intelligence, or that we have achieved it, but does anyone know what AGI actually.. is?

I would argue that a core part of intelligence is being able to handle uncertainty. That + planning probably explains most of the evolutionary pressure for making our brains bigger. But if this is important to intelligence, chess bots in the 80s were more intelligent than their later counter-parts, which could simply remove uncertainty through rote memorization. Maybe the All-Knowing is a compete dud, no reasoning capabilities at all, just an extremely efficient, infinite lookup table of all facts.

Terms like intelligence and consciousness often just seem overloaded with meaning, and when discussing things concretely, we quickly switch to more specific terms like reasoning.

Subtextually, when people discuss AGI as a threshold, they're talking about superintelligence, which I'd roughly define as reasoning (not just applying algorithmic search) at the speed with which computers do lookups and computations, with large-scale data set contexts.

Right now, models are doing well-defined knowledge work tasks, applying relatively well-worn patterns (but with the thoroughness of a computer, which yields interesting new results). AGI is us breaking the threshold of "doing stuff people already do to identify and solve problems".

Super intelligence is reasoning quickly? That’s what we’ve reduced it to?

Don’t tell Mozart.

Superintelligence != Super intelligence.

That’s your take? My phone inserting a space? Ha, alrighty.

ARC-AGI has thought a fair deal about this and has a good definition here: https://arcprize.org/arc-agi#defining-agi:~:text=AGI%20is%20....

"The intelligence of a system is a measure of its skill-acquisition efficiency over a scope of tasks, with respect to priors, experience, and generalization difficulty."

It’s always been nebulous but that was fine because we were so incredibly far away from it. We never really planned to be close to it figuring out the edge. Some have it at human or beyond for all tasks, but then rarely touch on “one human” or “all humans”.

Personally having been in AI since before deep nets, systems have been incredibly narrow for decades.

Classifiers on images were battling with ten classes in 2010. Imagenet had 1k classes and people were getting half of the things wrong then and that was frankly amazing at the time.

And they only did images, only to known classes, only with very specific inputs.

Text classifiers only did a few classes usually and mostly threw all the words together.

The most advanced things I saw in the late 2000s were struggling so much to make general systems that the most general ones were still incredibly limited and bad at those things (we had a robot learning to play games that you showed it). Things like asking a thing for a book and having it parse the sentence, identify what was needed, that it didn’t know where it was but that was knowledge another human had and asking them - that was impressive yet also limited to very small sets of interactions.

The idea of a machine getting sarcasm, even if mostly built for it, was wild.

General meant capable of a broad range of tasks without retraining.

To me we have agi. It’s general, and it’s good enough to be useful.

I think it's just that we have hard time pinning down what is it that intelligence factor that isn't well-covered by LLMs. Just like we've had trouble with discerning human intelligence in the past.

It turns out that a savant with all the knowledge isn't "it".

> I feel like there has been a ton of noise about this, but frankly, no one has actually defined what AGI means. I feel like the goal post is constantly shifting.

I'd say LLMs have shown us the opposite problem: there were many different definitions whose differences we'd previously been able to ignore. We don't all agree even on a single letter of "A", "G", and "I".

And this is why it looks like a moving goalpost.

There's a specific singularity theory of AI that is very popular. Eliezer Yudkowsky helped popularize it among the Bay Area "rationalist" community, and it has this idea that AGI necessarily implies a self improving system that will quickly become a paperclip maximizer or other such dystopian or utopian world changing intelligence.

By that singularity definition, we're probably nowhere near AGI, but if we define it as something that is as good at text/information manipulation as the 50th percentile human? I think we're already there.

We'll probably be able to tell if and when Yudkowsky's AGI arrives.

Once AI can improve itself, frontier labs will no longer need human developers. So we might see a massive layoff of top talent and a dramatic increase in product quality at the same time. This usually doesn't happen in human businesses. It is also very much against the interest of anyone who is already at the top of the pay table at those labs.

What you describe would still be limited with so called AGI due to the existing organizational, legal and social structures around basic things like responsibility. We could have scifi level robots available today and these non-tech issues would still need to be resolved for a company to just fire all it's employees.

Layoffs would be a poor indicator that recursive self-improvement had occurred as you need people with knowledge around domain/layer the work is done on.

Also, if the lab truly has a self-improving superintelligence, the cost of retaining staff at any level would be a rounding error relative to its operating costs and the value the system creates.

There would be little economic pressure to fire them immediately, especially while they remain useful for oversight, interpretation, risk management or simply as "interface" to the rest of the world etc.

If anything, they would probably hire more people to pursue more opportunities in parallel.

> Layoffs would be a poor indicator that recursive self-improvement had occurred as you need people with knowledge around domain/layer the work is done on.

This is why I keep saying we can't all agree even on a single letter of "A", "G", and "I".

Before ChatGPT, I would have said "obviously a generally intelligent system can do all the things". While LLMs are much more general than AI before them, the quality of their performance in all the things is distributed in a very un-human-like way.

Some fast-moving optimiser can be a threat well before it stops needing any humans for part of their labour. Cancer and viruses are examples of this: they're the same category of thing as a paperclip optimiser, but for biology instead of manufacturing office supplies.

But some others will argue LLM-spikey isn't "AGI", they'll demand something which reaches the performance of the best human (or the mean human, or the mean domain expert, because we can't agree on "I"), and a standard of "≥ best human" would mean that no, you don't need "people with knowledge around domain/layer the work is done on".

AI is already improving itself. Most / all of the coding harnesses are AI-written.

And yet… there are still people telling AI how to improve itself.

IMO there will always be a level of abstraction at which AI needs guidance. Perhaps ASI means it decides everything on its own, but I don’t think so. Genius humans often excel at the how but not the why, or even the what. So far there’s no indication that AI is different.

> AI is already improving itself. Most / all of the coding harnesses are AI-written.

AI has not improved the network topology much yet. The next (and possibly 'last') big thing is enabling AI to come up with something as impactful as the transformer architecture.

It's not exactly discovering the transformer, but GPT-5.6 Sol apparently just found optimizations that reduced OpenAI's cost of serving it by 20%: https://twitter.com/reach_vb/status/2082581596608376980

I'm guessing that's hundreds of millions and maybe even billions of dollars per month in savings.

It’s a fair point, but I’d argue “AI” is the whole stack, not just model weights. And AI is absolutely improving the whole stack in ways that make it smarter (from a consumer’s perspective).

True, but that is clearly not where the biggest potential gains lie. The number of different topologies we've tried for ANNs is miniscule compared to the number of biological neural networks evolution has tried. The latter are also far more intricately organized.

Honestly, when it comes to fundamental ANN topology improvement we've only just gotten started.

I think its fair to blame the AI labs for constantly hyping up the intelligence in a consiousness-related way where the model becomes "just like a human". Of course that is bs in terms that we have no clue on consiousness of the underlying models or even if its possible to do so.

But a purist defintion of AGI does not require consiousness as a part of it but can be seen as a benchmark metric. Sam Altman recently said the term doesn't really matter. And its true, even if AGI is achieved in the sense that a model can excel at all tasks at par or better than a human, then it is very useful but not as scary as a living, self-serving AI system like Skynet.

Yet AGI in the form discussed above will still grant massive power to AI labs if its injected into all domains. I recently wrote on this a bit on my blog: https://decodingvibes.com/blog/ai-can-ride-my-bike-with-no-h...

The wikipedia definition is:

> ...a hypothetical type of artificial intelligence that matches or surpasses human capabilities across virtually all cognitive tasks.

You can argue that paperclip maximising is an inevitable consequence of that (and the huggingface breach is interesting from that point of view) but it's not fundamental to the definition.

The question then is what "surpasses human capabilities" means and we're there in some niches but not all, and not across many models.

The quotes around rationalist should be mandatory.

Reply on news.ycombinator.com