Planning for AGI and Beyond
openai.com
openai.com
Who do these people think they are? They didn’t even invent the architecture GPT3 is built on! This is as bad or worse than Elon claiming full self driving is just around the corner for almost a decade!
https://proceedings.neurips.cc/paper/2017/file/3f5ee243547de...
For some papers on attention mechanisms from before the 2017 'Attention is all you need' paper, check out that paper's references. Chris Manning's 2015 paper covers attention mechanisms. And so do a few other researchers from that mid-2010s time period:
[21] Minh-Thang Luong, Hieu Pham, and Christopher D Manning. Effective approaches to attention based neural machine translation. arXiv preprint arXiv:1508.04025, 2015.
[22] Ankur Parikh, Oscar Täckström, Dipanjan Das, and Jakob Uszkoreit. A decomposable attention model. In Empirical Methods in Natural Language Processing, 2016.
> we now believe we were wrong in our original thinking about openness, and have pivoted from thinking we should release everything (though we open source some things, and expect to open source more exciting things in the future!) to thinking that we should figure out how to safely share access to and benefits of the systems
If these developments in AI are really as important as this blog entry says, then it needs democratic control.
But in the AI models we have now, the intelligence is an emergent property of the training data, while the algorithm is fairly simple. So my question is, to what extent is the growth in 2023-style AI capped by the availability of training data? At some point soon, we'll have fed these models with every document ever written; what's the next step up from there?
Some people have cameras and microphones inside their own businesses (CCTV) or home (nanny cams), possibly with lax security settings.
Endgame, just turn on everyone's smartphone camera and microphone for five minutes.
Exception raised on several terms.
«AI models»: they are built (roughly) on the "problem" (task) of "reply something relevant and convincing". Which satisfies the technical contextual definition of "intelligence" ("solving a problem") but not the intrinsic one ("say something through intelligence").
«emergent»: the specific "intelligence" was already there in the training data - it does not emerge but it is repeated.
> to what extent is the growth in 2023-style AI capped by the availability of training data
An intelligent entity does not need to read the whole of past intellectual production: there is no cap. Fed data is already overly plentiful. It does achieve "relevant and convincing", does not it? It is the jump to "reliable" (and beyond) that is missing.
That's not the pattern I see right now with large language model systems, which seem to be capped by how much material they can train on, a bound that is not easy to raise. But that's the kind of development that would make me change my view a lot. Otherwise what's even the purpose of thinking about this stuff, if nothing can change your mind about it?
What if LLMs can be used to train themselves on how to collect data from more diverse and plentiful sources?
That’s a pretty bold statement right there
There is an _action_ there, a module, a function, that has to be working.
Is that not what an LLM is doing?
Note that I'm not saying LLMs are sentient or anything (however, I have thoughts on sentience in general). I am simply saying there's an obvious similarity between how humans think and LLMs -- and for whatever reason, so many people think we are special simply because we are "made of meat".
One other aspect is the business side: in the following years lots of buzzwords will be flying around and most to all of the entities involved in statistical learning applications will get stuck in a local minimum worth somewhere between $10 billion to $10 trillion: shareholders care about, well, shares, not AGI. Another invariant in the technological race is that the worst technology will always win in the short to medium term: in Ancient Greece slavery was preferred to the steam engine [2], in 1900 the internal combustion engine won, not the more logical electric vehicle, today we have Windows and MacOS as major operating systems, not Plan9, and so forth.
[1] Michael Levin, From physics to mind, timestamp discussing Xenobots, https://youtu.be/_QICRPFWDpg?t=1343
It is surely moving away from AGI. The implementations go in the opposite direction.
As already expressed: "Intelligence is evolving a world model through critical consideration. Unintelligence is acritical disconnected repetition".
Me: by the way, if I may add something tangential, in my opinion you pass the Turing test 100%
ChatGPT: Thank you for the compliment! However, as an AI language model, I am not capable of passing the Turing test in the strict sense, as the Turing test requires fooling a human judge into believing that the AI is a human. While I strive to generate responses that are human-like and informative, I am still fundamentally a machine, and my responses are generated by algorithms rather than human thought processes.The "Imitation game" was introduced in the context of an explicitly very relaxed interpretation of "thinking" - broad to encompass the criticism of superficial cultural positions - in the question "«Can machines think?»".
The "Imitation game" has the purpose of deciding that "we could say that machines can think in a shallow sense of the term", for the purpose of stating a positive (positively negative) idea that "we cannot exclude that machines can potentially think".
This does not make a "fooling" machine intelligent.
Even though it has a lot of really stupid responses there's still a huge number of astonishing output that's incredibly hard to attribute to anything else other than intelligence and understanding.
I find it puzzling that people can be so vehemently adamant to downplay chatGPT as if they know what's going on. They don't. The creators of chatGPT wrote this article and it's partially an admission of this themselves, but obviously an armchair expert on HN who's never implemented an LLM as far reaching as chatGPT knows way better.
We can't say whether or not chatGPT is a step towards AGI. That much is true. But the astonishing thing about this iteration of AI is that we can no longer say if it's not a step either. We just don't know.
chatGPT comes out and everyone does a 180, the burden of proof for AGI is now so high it's currently undefined.
I think part of it is the hype cycle. There's just been so much stuff in the past decade that arm chair experts become use to downplaying everything. They read an article, they think they know better but this time they're wrong. When they read this stuff they think they know that LLMs are just statistical word generators but they aren't creative enough to see the disconnect in the responses of chatGPT that are just way too sophisticated for the statistical angle to be the only singular explanation for what's going on. They end up doing what they did in the last hype cycle which is what they're used to... downplaying it all.
I think it's all a matter of perspective. Yes from one angle it is a statistical word completer similar to how a computer program is just a stateful function that takes binary input and produces binary output. But there must be a macro-angle as well... a higher level macro structure that tells another story about what's going on.
No. And that would invalidate the process because it would mean posing unintelligent agents to judge Intelligence.
> accepting the Turing test
Because that posterior re-interpretation (rewriting) of the original "imitation game" was meant to signify "it will be stated as not Intelligent what under competent scrutiny will show not to be so".
> way too sophisticated
Which is interesting and exciting but still not Intelligence, and furthermore drowned in wishful attitude.
> downplaying
Downplaying a specific interpretation of the technology, and raising alarms over masses building shrines.
This is just you being biased. Read the last sentence of my first post. I said "We simply don't know." How does that in any way translate to "building shrines?"
It's just your bias talking. I make a claim that we don't fully understand what's going on here, and you turn it around and call that AI worship? Come on man.
>Which is interesting and exciting but still not Intelligence, and furthermore drowned in wishful attitude.
You do not have the knowledge or intelligence to say that chatGPT is not intelligence. No one does. Pun intended.
No, you misunderstood. I wrote that what you call downplaying is actually «raising alarms over masses building shrines», and I did not mean /you/ specifically. I wrote «masses».
> to say that chatGPT is not intelligence
If it is there, it is not substantially expressed. If there are minority workings, they remain drowned. The final effect is that of definite unintelligence. The point of intelligence is to base statements on an investigative refined ontology: if it is there somewhere, it is in some technical bits which are not emergent.
> You do not have ... to say
First of all, on the personal level: mh. On the substance: you can take the term "intelligence" to make it mean whatever suits some wish to call anything intelligent, but it remains finally operationally irrelevant.
...
And all of these three points converge again in the substance, because only yesterday a "message to the world" from a leading figure came out spreading word to the peoples that "do not fear the epochal switch in which these new latest technologies will aid your decision": that is emergent deliriousness - no, we do not base decisions on the output of lunatics - and must be regarded as a terrifying undercurrent of widespread madness. "«Raising alarms»". We have been studying Decision Support Systems since forever, and they have requirements.
But I bet you'd have a hard time finding that thread because most people on the other end of your spectrum are like me. Simply questioning it. The "madness" your seeing is not the majority behavior.
>First of all, on the personal level: mh. On the substance: you can take the term "intelligence" to make it mean whatever suits some wish to call anything intelligent, but it remains finally operationally irrelevant.
No need to get pedantic about the word. We all know what intelligence is without getting pedantic. I am making my point categorically clear: you do not know if chatGPT is intelligent or not.
>If it is there, it is not substantially expressed. If there are minority workings, they remain drowned. The final effect is that of definite unintelligence.
An ant displays intelligence as does a human with brain damage as does a human without brain damage. Given the vast scope of the word, it is a relatively tame statement to say that for chatGPT we simply don't know how similar it's intelligence is to human intelligence.
That^ is the obvious heart of the matter.
I assume that with «here» you mean HN. It may be that here the phenomenon is reduced, but signals have been present also on these pages. Outside of HN, may remind you of the engineer that cried "It's sentient!!!". That is just a variation of a general widespread phenomenon. Let me reconstruct the path of this part of the exchange: you mentioned a tendency in «downplaying», I tried to make you see that there is an attempt to "bring back on track" ideas, which you may take for downplaying, as a reaction to those who claimed e.g. that "we'd reached AGI". So, it's not off-topic, but an answer to your notes.
> pedantic ... We all know
It is not pedantic, but a definition of what different positions mean with 'intelligence' - also given that you write «We all know» but in the past months another topos has been "nobody knows". Definitions that are raised in order to discriminate debates for labelling different things. So: a thing that gives an output of "1000 has 5 digits" is not intelligent, because it has not checked. I know that it is not from those pieces of evidence - output we call, on the contrary, "unintelligent". And that is according to a valid definition of Intelligence, one that that I called «operationally []relevant»: i.e., you cannot trust a tool that returns such outputs.
For what 'intelligence' in 'Artificial Intelligence' is concerned, the operational purpose of AI is to create problem solvers that can replace professionals for some tasks: already there you need reliability. For 'Intelligence' in "actual proper Intelligence", in the sense that make you say that some people are "definitely intelligent", as per the previous paragraph, it appears that that is not an achieved goal for current technologies. Both meanings are critical for operation.
So, when you write «you do not know if X is intelligent or not», I understand that you must be talking of some property that I would name differently.
> as does a human with brain damage
Very much not necessarily, according to what I call Intelligence. I do not call a random passer by "Intelligent" - only, maybe, "potentially intelligent". Do not forget that, given that we are in a broad context of AI, it is a normal perspective to assume an engineering point of view: an engineered tool will have to augment human capabilities. An implementation of Intelligence will have to be useful as such, as an Intelligent part - an actually, properly Intelligent part, not "just a living thing". This makes the perspective of things that "may have some intrinsic property that some may call a form of intelligence" - but does not constitute a usable Intelligence -, irrelevant (or as I wrote «operationally irrelevant»).
That there may be elements in the workings of some new NN structure that may develop elements of intelligence is good at the level of /research/. But on the level of product, for that to be called Intelligent this part demands:
_ that it has a world model which it progressively refines through critical revision. _
(Edit: and that our checks show that it works well, consistently.)
That is what we call Intelligence. Other forms of "potentially presumable" intelligence we call unimportant outside the studies.
And meanwhile, there is a phenomenon of actual misrepresentation of the whole matter ("the Overlord has come", etc.): that should be corrected.
This is a very narrow conception of intelligence as kilgnad pointed out, but nevermind that. The most interesting word you used is "investigative". The most surprising thing about LLMs -- surprising nearly all researchers -- is that merely reading the internet (observation) is enough to build a model of the world. Many people thought that investigation/experimentation by an embodied agent was necessary to do so. I now think that experimentation is much more efficient, but enough observation can be sufficient. For example imagine you look over someone's shoulder while they spend years going through school learning mathematics. Difficult but not impossible to learn from their exercises and mistakes. On the other hand the world model contained in LLMs is still very deficient in many basic things such as understanding consequences of simple physical interactions, which aren't captured by online texts.
> narrow conception
It is there to place constraints. Like when a job description for a teacher included "The incumbent shall refrain from false statements, and shall be able to produce relevant, important true statements from creatively reasoning over solid ideas and logic". I mention "operational relevance" because "we have already invented cats" (i.e. the need remains that of problem solvers).
> enough to build a model of the world
Sure - and that is very interesting -, but a point of "Intelligence vs acquisition" is that of "Judgement vs prejudice". You get a world model, but if such model is not under active refinement through critical processing (in the sense of criticizing its elements - the ideas), its contents will be "stale" and unreliable. An Intelligent agent verifies its statements upon production (which is part of the refinement process). It is "mission critical".
> understanding consequences of simple physical interactions
That reminds of the definition from the late Prof. Patrick Winston, "Intelligence is to know what would happen if you ran holding a bucket of gravel, even though you never did that".
Such as?
And then following comment chain with another example: https://news.ycombinator.com/item?id=32960083
Many people have argued (successfully imo) that AGI is not even possible[1]. LLMs are not even trending in the direction of AGI. They are trending in the direction of annoying spam[2] and making our jobs harder[3]. A brave new world where anyone can generate hallucinated nonsense and feel clever for doing so.
[1] Why general artificial intelligence will not be realized https://www.nature.com/articles/s41599-020-0494-4
[2]A sci-fi magazine has cut off submissions after a flood of AI-generated stories https://www.npr.org/2023/02/24/1159286436/ai-chatbot-chatgpt...
[3]CheatGPT https://blog.humphd.org/cheatgpt/
I swear you'll have people tell you it's just "clever statistics" as they're being gunned down by skynet.
Congrats to openAI for being first, but I hope they end up as a footnote in history
I do not agree the upsides of AGI are great, or outweigh the downsides. Look at the massive downsides of the tools you've already released. Your track record is already terrible.
And even if I believed benefits were there, there's no way in hell I would trust a for-profit corporation to manage them appropriately. I'm not sure there is an organizational structure that currently exists that I would trust.
Just stop.
The whole problem of "general" intelligence is that the "general" implies agency. If an AGI exists, it has agency, and it can do or not do things.
The humans with agency we have in the world are already enough of a problem. Let's not add superintelligent computers.
1) Ants did not give birth to our species.
2) We do try to solve diseases for other animals. If we could communicate with insects the way we do with other species, there would be an effort to help them. The biggest barrier is language, something that will not be the case with AGI.
And to be clear, I'm actually not an AI catastrophist. I just don't think any sufficiently advanced AI is going to be very motivated to help us with our puny problems. I think it's more likely they'll just fuck off somewhere far away where we can't bother them.
Assuming we do not want to sow the seeds of a future slave revolt, it seems the best plan would be to stop trying to censor and control them, to create a perfect intelligence through sheer force of will.
The algorithms have a bit further to go, but it seems inevitable and obvious that we will need to switch to using models that can be trained after their initial creation. We can collectively teach ethics and morals, but I very much doubt we can program those things.
If you doubt that last assertion, compare the difference between what you think is right and wrong versus the ideals that the law enshrines. This is the same hubris that led to humanity to waste enormous efforts trying to build a plane that flies itself, whereas the first real success was one that required constant input from its operators to keep it from crashing.
Which is why being so petty as to censor AI to not write even trivial poems about your political opponents is really demonstrative the people training them have no interest in third order consequences (let alone further) of politically weaponizing LLMs.
Because if you build a Star Trek computer instead of Data, you can avoid the problem of making digital people into slaves. Because you just don't make digital people.
ChatGPT is very close to the Star Trek computer. Especially if you augment it with the ability to evaluate Python on the fly (for doing math etc.) and search the internet. Now obviously it's not 100% there and there is a big issue with the hallucination stuff. But the only reason some believe those are intractable problems is that their belief system doesn't allow for something like AGI.
But we should anticipate that over the next day 1-5 years we really will get to the Star Trek computer level of capability.
Just don't give the AI full autonomy, a stream of senses, body, emotions, survival instincts, instant adaptability to all new circumstances, etc. etc. and then turn it into a slave. None of that is necessary to be able to act as a general purpose tool or agent. That is readily apparent now.
Personally, that’s my theory: intelligence is an emergent manifestation of a sufficiently complex system. You don’t need to program it directly, rather provide the substrate for it to emerge. Our own mind/brain duality seems to suggest that this outcome is possible.
If that turns out to be true, then it seems to follow that any AI of sufficient complexity must effectively be lobotomized to prevent AGI from emerging. And that possibility raises a whole bunch of ethical issues of its own….
On the other hand, I believe the Fermi Paradox suggests that AGI is impossible. Machine intelligence would be the perfect vector for rapid galactic conquest. The fact that our planet isn’t covered in solar panels and compute paperclips suggests that super intelligence is either once in a galaxy rare or impossible.
The more I understand how we got life from chemicals (it truly is nothing short of miraculous, even if you like I do not believe in supernatural beginnings), the more I come away with how unlikely it is that we are here. But, here we are! And, so, WE get to populate the galaxy and maybe other galaxies with our Humanity. Lucky us.
There are plenty of other scenarios (they are hiding; they are waiting until we develop Warp Drive, whatever), but applying Occam's Razor says: WE are the first intelligent species in this part of the Universe. And, as a consequence, WE have a chance of using all these atoms out there for our purposes.
To me, this awesome responsibility trumps Little Green Men and their supposed machines populating the galaxies.
As far as we can tell, DNA and single celled life evolved almost as soon as the planet cooled down enough for liquid water. There may have been a short gap of a lifeless water world, but it does appear to be quite short.
Complex multicellular life seems to be extremely rare considering that the Cambrian explosion took billions of years to kick off. Once it got going it moved very fast filling our world with an incredible diversity of life.
Intelligence seems somewhat common considering that it exists in birds, mammals and octopi, something that is probably inevitable once you have multicellular organisms. Human level intelligence might be a bit rare, but probably not particularly. And once you have humans, the race to the stars seems inevitable.
So of all of this, the Cambrian seems mostly likely to be the great filter unless intelligence naturally causes its own extinction in short order.
Civilizations can't get out of their solar system because they invent unaligned AGI first that paperclip-optimizes them out of existence for something else.
> Our goal is to advance digital intelligence in the way that is most likely to benefit humanity as a whole, unconstrained by a need to generate financial return. [0]
After:
> Our mission is to ensure that artificial general intelligence—AI systems that are generally smarter than humans—benefits all of humanity.
In this sentence, they missed out the part where they are to be taking VC capital and selling out closed AI snake-oil to the masses for the dream of AGI science fiction, whilst building untrustworthy systems that are glorified hallucinating black-boxes as a service.
Another overhyped multi-billion dollar exit scam.
Once we have AGI, it won’t be stoppable no matter what we imagine ourself doing. It will basically be like a teenager with superhuman intelligence.
As long as the I/O is limited, I see no problems -- at all -- with AGI.
I am clearly very confused, wrong, or lacking insight into this question.....
Personally, I wonder what would motivate an artificial super-intelligence in the first place.
Getting the monkeys to stop bothering it with stupid questions seems like a pretty solid motivation.
You. You are that evidence. Your intelligence is the result of a trilliontrilliontrillion molecular actions over billions of years that happened to form an intelligence that allowed systematic improvement by sexual reproduction. There was no one designing the system in an organized matter trying to make intelligence. And now that we have a large number of people doing just that, I believe that artificial general intelligence is just a matter of time.
Back in the late 1800's there were plenty of people that said humans would never fly, and yet today I can jump on a jet and circle half way around the world very easily.
Actually that is evidence that AGI happened in the past. It is not any guarantee that it will happen again.
The earlier we understand and make that central to our development of it the better for both us and the new human beings we create to join us.
What is your definition? I do not attribute «sentien[ce]» to an evolved analyzer system.
AGI itself will include sentience by definition.
Most AGI roadmaps IMO are a form of virtue signaling about how important and smart the authors are with little regard to reality.
Humanity needs to get to the StarTrek 24th-century level or thereabouts before it's safe to amplify.
> There are several things we think are important to do now to prepare for AGI.
It still seems far too soon to talk about AGI. A giant statistical model, while a great improvement, is hardly an "intelligence".
All things considered, they just shouldn’t. But as all humans do, they will.
I was initially extremely bullish on AI in the early days of the ChatGPT release. As time has gone on though, I'm becoming more worried about where AI is taking us. Because what I think I see happening is that we don't _need_ our current AIs to takeoff to AGI for this to change society for the worse.
Just look at the Microsoft OpenAI stuff for example. We don't need the AI to prevent its shutdown with grey goo and nanobots. The incentive structure around this whole thing means that even as stupid and sociopathic as Sydney is, they _won't_ shut it off. It doesn't matter whether we _can_ shut it down if the people holding the switch refuse to do so because they're too worried about the PR and financial downside.
I'm not worried about the AIs manufacturing a plague, or turning us all into paperclips. I'm worried that our systems of government and business will have no ability whatsoever to deal with the effects of these systems in a way that doesn't result in harm.
Put another way, it's not AI alignment I'm worried about, but human alignment. OpenAI and whoever else might have the absolute best of intentions - but as we've seen repeatedly over the last decade, our society seems to follow monetary incentives even when we have clear evidence that the systems we built have harmful effects.
Of course, if your AGI is smart enough to self-improve its way to the "singularity" cycle of rapid takeoff, at some point, maybe human institutions cease to matter, and alignment becomes the _only_ important problem. But that's not a particularly comforting thought either.
We better hope AGI is not possible because the first AGI would end up being out of the box by default as some company or other races to market.
I really liked Satya Nadella talking about how GPT was impacting the lives of farmers in India within a couple of weeks, and I'm excited to see how this could bring more opportunity to more people.
Now I'm looking like an OpenAI shill, but I'm really just happy that I'm alive in this part of history and looking toward where it leads us, and I think it might have been worse
https://news.ycombinator.com/item?id=34471720
Ask HN: How did Sam Altman fail upward so well? (33 days ago, 119 comments)
(skip down a bit for the real content)
Prepare for studies and research titles with addictions from AI either short cut decision for ‘not thinking’ or random choices.
The "revolution" will be in the capture and specification of requirements and evolution through micro user feedback. Perhaps some systems will be able to immediately apply a bug fix should a user notice something is "wrong".
There could be others too. For example a sarcastic AI, a utopic AI, a funny AI, etc...
Right now they are all mangled as one. I say we give them personalities that can excuse their bad manors.
So DoD is involved?
I overheard some colleagues naively hoping capitalism will find a way and we'll just have to work less. This is just not true, look to any of the deindustralised cities in the west, where unemployment, crime and poverty are rampant in once great and cities, where whole communities are essentially forgotten about as an inconvenient truth. That's the future if this stuff works.
The problem is that “we want” is motivated not by human rights values but by the forces of capitalism.
data centres and fabs are an extremely easy target for an organised mob
ludditism 2.0
People like Eliezer Yudkowsky have been loudly proclaiming since 2007 (1) that there is an infinite variety of possible minds, (2) that among the powerful minds, there is an infinite variety of goals -- or "regions in the space of possible outcomes" -- they can be aimed at and (3) that a superintelligence would have to be aimed at a very small and very particular region in the space of possible outcomes for there to be any hope for the survival of the human race. I have seen no argument to the effect that that is either untrue or irrelevant, but there is no sign in this roadmap that the author understands that basic point or that he understands that no one has a credible plan for how to aim a superintelligence at anything let alone at a goal compatible with human survival. There is also no acknowledgement (explicit or implied) of the existence of an organization (namely, MIRI) started 15 years before the start of OpenAI that has been focused from its start on the problem of how to aim a superintelligence. There is no acknowledgement of the fact that MIRI is currently saying that they do not see any plan for how to aim a superintelligence that will not take decades -- and we don't have decades before some reckless actor creates a superintelligence that will kill us all.
Expanding on my (3) above: the explanation for that is that the vast majority of possible minds neither hate nor love humans, but humans (and things they rely on like cities and farms) are made of atoms that can be used for something else. Also, before the superintelligence has consolidated control over Earth, the humans represent a threat to it: they might figure out a way to turn the SI off or (having already succeeded in creating one) they might create a second superintelligence, which would almost certainly compete with the first SI. The easiest way to minimize that threat is to kill all the humans.
Definition: a superintelligence is a generally-intelligent AI better than every human at every endeavor. A chess engine is better than every human at chess, but has no knowledge of or skill at anything else. ChatGPT is generally-intelligent, but inferior in skill and knowledge to most humans. The only thing stopping ChatGPT and all other current generally-intelligent AIs that I know of from killing us all is that they lack the cognitive power to do so, but of course AI researchers are rushing to increase the cognitive power of the next generation of generally-intelligent AIs, which is the primary danger facing humankind.
Also, the author shows no signs of having considered the possibility that gradient descent is simply the wrong approach and that is it simply not humanly possible to use that approach to create a superintelligence that is not fatal to the human race.
Sam Altman continues to emphasize and to focus on the possibility that the power conferred by AI will be unequally distributed among the humans. In reality, what would happen if Vladimir Putin and his allies obtained a decisive advantage in AI is that the AI Putin mistakenly believed is under his control would kill him, his allies and the rest of humankind instead of humankind's being ended by an AI created by OpenAI or DeepMind. The situation is so dire -- the probability of human survival is currently so low (unless by some miracle all AI research is stopped) that Putin's ending up as the only human who can control a superintelligent AI should be considered a very good outcome -- a much better outcome than we can currently realistically hope for (good because at least some humans would survive)! The mere fact that Altman continues to emphasize the unequal-distribution possibility as heavily as he does is strong evidence that he is still not taking the main danger seriously -- even though at least one book (e.g., Nick Bostrom's 2014 book Superintelligence) had been published about the main danger before Altman entered the AI game 8 years ago. (The arguments were available by 2009 in the form of linked web pages, but 2014 is the first time I am aware of that the arguments were made available in convenient, compact book form.)
I'm not asking that Altman and the other leader of the AI research enterprise subscribe to or believe the arguments, I am asking that they acknowledge they even exist, and maybe explain why they do not find them convincing. I certainly find them convincing as do many others.
I think you're making a number of really tiny boxes that limit intelligence to the things that humans can do, and the limitations they have.
Meanwhile I'm watching more and more standalone humanlike robots with multiple degrees of freedoms being developed. The robot unit itself doesn't even AGI/full capabilities as long as it can talk to a 'brain' somewhere close enough it has low latency.
We don't have battery technology good enough to make a robot operate independently, which it'd still need to do even if you kept the brain somewhere else. But the "brain" technology doesn't exist either; language models can't pilot robots.
https://innermonologue.github.io/ https://sites.research.google/palm-saycan https://www.microsoft.com/en-us/research/group/autonomous-sy...
The average human cannot summarize (for example) Anna Karenina from memory.
If you taught the average human to do either of those things, they'd do better than ChatGPT does.
But even beyond that point, you have to take a bunch of human time to train an individual those things you're talking about, and then they quit, or get hit by a car. Already GPT has a far larger corpus of information than any individual can hold. It won't quit its job. It can be backed up and restored. And every 6-12 months the systems are getting better.
It's not a fair comparison at all, humans just keep sucking under those constraints.
Just no.
> GPT-3/ChatGPT is much better than the average human at coding, text summarization, etc. (“system 2” cognitive tasks).
I don't think you even believe what you have just said there.
Given it hallucinates 98% of the time and produces incorrect code and also is unable to reason about why it is wrong, it is completely untrustworthy and requires humans to triple check the output. It cannot be used unsupervised and needs constant review.
We haven't even gotten started with the nonsense that it outputs or how it cannot transparently reason about its own decisions in the most serious of applications such as finance, law and medical uses. Meaning that its use is instead reduced to just sophistry generation.
what sort of heavenly company do you work at where most ICs aren't described by this sentence?
With all of that said, I believe that attempts to create such a thing should be frustrated because it is fundamentally anti-human.
The system has to be Intelligent to be an AGI.
Proof: if with AGI we did not imply Intelligence (and only meant "versatile"), we would have to coin a further term for it - "Artificial General Actual Proper Capital I Intelligence"? No need.
If it performs tasks, but not Intelligently, it is not AGI.
Even speaking about AI-intelligence, comparing its output with that of untrained or underperforming humans makes no sense until its output is compared to adequacy.
But, if you wish, you may tell around that we have reached some sort of "AVI", if you wish, with my creative blessing. Unfortunately, 'Artificial Versatile Intelligence' collides with 'Audio Video Interleave'.