There is no A.I.
newyorker.com
newyorker.com
Look up the definition of "artificial" while you're at it.
However this is a great simplification, and borders on an absurd reduction. You can model our brains using linear algebra, however that doesn’t mean our brains are linear algebra computer. There is a whole lot more going on than neurons receiving feedbacks from other neurons which adjusts the weight for subsequent firing. A lot of our behavior is actually inherited (I know I spent a whole week here on HN arguing with IQ advocates on the nuance of that statement), neurochemicals and hormones add a whole another level of statefulness not seen in artificial neural networks, the brains ability to make computations is actually pretty limited (especially next to a GPU). I mean, cordiseps exists, meaning a fungus can infect an organic system and control its behavior, there is 100% chance that some yet to be discovered viral and bacterial agents, not sharing any of our DNA—and certainly not “connected” to the “weight matrix”—are also influencing our behavior (just not as dramatically), and there is 100% chance they interact with our DNA also controlling our “innate” behavior.
What is going on in our brains can only be modeled using statistical ordering of semantic words and actions. The real world brain is always going to be infinitely more complicated than this model.
And all of our surprising wins and awful mistakes had explainable reasons, dammit; it wasn’t just a misfiring of trained statistical networks!
That’s still the hole in my theory of consciousness. I admit it ;)
But it doesn’t give me as much cognitive dissonance as others to believe that the process of performing moral actions still has to be “processed” by me, a processor. In some sense.
The LLM value add for coding is less than the value add of syntax highlighting in my experience.
In my experience and it's certainly useful for unit tests speeding things up for me pretty dramatically.
Also when working on a new development language was quick to point out "how you do that" with the inline vars I needed manipulated vs look ups in Google and copy pasting.
To solve ourselves is to know ourselves completely, and to know ourselves completely is to be honest in who and why we are what we are simultaneously across all persons. It assumes perfect knowledge.
There is no statistical approximation nor computational power which can do this.
> People are understandably desperate to understand their experiences as more than an encoding of a thing that might be explained.
Another way to frame this is, "some people are nihilists and do not see life as more than an encoding of a thing that might be explained."
To know anything (an X) completely you need perfect knowledge. Hence people come up with a set of simplified ways of reasoning about X. They call this a model of X.
Model is incomplete and so primitive, so dumbed down, that we manage to play it forward/backward in our heads or our computers.
If a model checks out with the real outcomes we proudly exclaim that we understand X.
I'm not being sarcastic, that is just a real method we use all the time.
There may be a part of the brain that is modelled well by an LLM, but if so, there seem to also be parts that aren’t, or even existing “multimodal” models like GPT-4, which is more than an LLM.
Humans, including their brains (both in the narrow sense and in the broader sense of “everything that contributes to cognition”, which may extend beyond the brain proper) are machines, and their function probably will, someday, be mimicked by machines. But they are still more complex machines that modern “AI” systems
And that's completely fine.
Not just that LLMs are very capable by themselves, but significantly improve speech and image models.
We’ll need to break out of the Chomsky hierarchies and develop some new theories of language.
Vocal communication isn’t unique to humans, a new theory should be more broadly applicable to non human and non vocal languages.
I’m excited to see what big questions will be answered, and what new questions arise.
That's highly debatable. For starters, who said there's no creativity or inspiration in humans, or for that matter, that there can't be in a complex A.I.?
How we achieve that creativity or inspiration is irreleant, as long the entity (human or AI) showcases creativity and inspiration.
Nor is it much clear why all being "only (a very complex) agency" would preclude creativity and inspiration.
That's like a worse version of "a human can't be creative or have feelings because it's all a bunch of molecules".
There is such a thing as emergent properties.
>And that's completely fine.
That's also highly debatable. I mean, that it would be "completely fine" if you were right and there wasn't "creativity of inspiration".
There has been a long history of reifying the behaviors of living things in general (e.g. vitalism) and humans in particular (e.g. dualism).
The success of LLMs challenges a lot of philosophy dealing with what behaviors are and are not possible in the absence of these categorical districtions.
I have had casual debates years ago, in which strong dualists asserted that the kind of creativity exhibited by today's LLMs is simply impossible. No doubt those folks are busy inventing "special philosophical creativity" that LLMs "aren't really doing," but they've lost credibility.
LLMs have demonstrated that there was never any need to invoke categorical districtions between human behavior and math-as-implemented-by-physics. The gap is closed, there is no more room for gods.
It is telling that you talk about all those things happening in "real time." Ask any super-regarded philosopher, from Plato to Wittgenstein (yes I'm excluding Dennet et al), and that would be quite the hoot to point out.
Yep. Elevators used to be proof that machines can think. Then compilers, and chess, and go, and search, and …
The problem with AI is that as soon as it works, we stop thinking about it as “artificial intelligence” and it becomes “just automation”. Then AI moves to the next goalpost.
most brains I run into don't do that much at all, mostly just existing and adaptation-execution
Our minds are in fact same statistical models with a gradually declining ability to learn and driven by exogenous irrational goals to eat and mate.
To crank it up a notch, the assigned task could involve generating itself subtasks which are handled in the same manner. This subtask generation could start to look a bit like will/intentionality.
Now consider if the top-level task is something like maximizing money in a bank account that pays for the compute to keep it running :)
(IMO this is still missing some key pieces around an emotion-like system for modulating intensities of actions and responses based on circumstantial conditions—but the basic structure is kinda there...)
It's just hard coded, whereas GPT's is dictated. More or less anyway.
Also our "fitness function" or motivations & goals aren't even that hard coded. You can easily modify them through drugs.
I'm not sure that's as clear cut as you make it sound.
For example curiocity could be a fine engine for motivation as well, even if you don't care if you'll survive or not.
- If you mean "you cannot EXPLAIN AGI/consciousness/intelligence if you don't understand it", then that's true, but it's a trivial tautology.
- If you mean "you cannot DEVELOP AGI/consciousness/intelligence if you don't understand it" then that's very debatable.
Historically we have been able to develop all kinds of things, despite not knowing how they work. Tinkering and trial and error is often enough.
After all that's how evolution solved the problem of creating consciousness/intelligence. There wasn't some entity that "understood" intelligence that created it.
Well that's not true. We tamed fire before understanding combustion, friction, heat, or anything else.
In fact it is damage to the prefrontal cortex (which has nothing to do with speech) which is mostly correlated with a detriment in intelligent behavior (suspiciously also social behavior; a food for though in what we consider “intelligence”). Victims of lobotomy had their prefrontal cortex destroyed, and their injuries resulted in them loosing their personalities and loosing basic function as human beings, even though they were still able (but perhaps not always willing) to speak and comprehend speech.
What you have that LLMs lack is a visual part of your brain - one which can instantly count quantities of objects up to about 7. That gives you tools that can be trained to do basic arithmetic operations. Although you have to be taught how to use that natural capability in your brain to solve arithmetic problems.
And of course for more complex things than simple arithmetic, you fall back on verbalized reasoning and association of facts (like multiplication tables) - which an LLM is capable of doing too.
Poor GPT though has only a one dimensional perceptual space - tokens and their embedding from start to end of its attention window - although who’s to say it doesn’t have some sense for ‘quantity’ of repeated patterns in that space too?
That has been my best analogy so far.
They'll never say I don't know and bullshit you into oblivion while never backtracking.
When you discuss other things, it goes off the rails a lot more. For instance, I have had it quote some passages of classic English poetry to me, stating blankly that those passages contain certain words. The passages did not contain any traces of those words or even remotely similar words. In that situation, GPT4 was being dumber than /bin/grep, which can confirm that some piece of text doesn't contain a string.
GPT4 is deliberately trained as a coding technician, so that it will convince programmers who will become its advocates.
1. Programmers are vain and believe that everything they do requires great intelligence. (Not just some of it.)
2. GPT4 can "talk shop" with programmers, saying apparently intelligent things, perform complex refactorings, and intuit understanding of a piece of code with minimal context based on meaningful identifier names, and so on.
3. Thus, some programmers will falsely conclude that it's highly intelligent, like them.
To be sure, what the technology does is unmistakably a form intelligence. It solves complex problems of symbol manipulation and logic, in flimsy contexts. To deny that it is artificial intelligence is silly.
AI /= AGI
That's not where the goalposts are for the definition of AI.
Computers playing chess is AI. Google Assistant is AI. An expert system from 1990 is AI.
no, it solves the assumed-to-be complex problem of constructing probable and believable text given a prompt.
Are we all vain? I guess we are, but I find this take to be a bit too simplistic.
I do agree programmers are arrogant and think everything they do requires vast amounts of intelligence.
I’n not sure what your point is. I am honestly curious. You think having coders as major advocates for your product is somehow a brilliant strategy? If anything, this will work against it.
For instance, as an exercise, I had it write a maze-generating Lisp program that produces ASCII. I wanted it in a relatively little known Lisp dialect, so I described to it some of the features and how they differ from the code it generated.
For instance, GPT4 hypothesized that it has a let which can destructure multiple values: (let ((x1 y1) (function ..)) ((x2 y2) (function ...)) ...).
In plain English, I explained to GPT4 that the target Lisp dialect doesn't have multiple values, but that when we have a pair of values we can return a cons cell (cons a b) instead of (values a b). I also explained to it that we can destructure conses using tree-bind, e.g (tree-bind (a . b) (function ...) ...). And that this only takes one pattern, so it has to be used several times.
GPT4 correctly updated the function to return a cons cell and replaced the flat let with nested tree-bind.
At the end of the chat, when the code was almost working, I made a bugfix to the function which allocates the maze grid, to get it working.
I told GPT4: this version of the grid allocation function makes it work. Without it, the output is almost blank except for the vertical walls flanking the maze left and right. Can you explain why?
GPT4 correctly explained why my function works: that the function it wrote shared a single row vector across the grid rows, giving rise to sharing. It explained it like a computer scientist or seasoned developer would.
It's like a somewhat dim-witted, but otherwise capable coding clerk/technician, which talks smart.
With GPT4, you're Sherlock, and it is Watson so to speak. (Sorry, IBM Watson.) It can speak the language of crime investigation and make some clever inferences. In the end, you do all the real work of designing and debugging, and judging complex requirements against each other in the broader context. It saves you the keystrokes of doing the tedious coding you're used to doing yourself.
On the other hand, you expend some keystrokes explaining yourself. Some of the chat could be saved and used for documentation, because it captures requirements and why some things are done in a certain way and not otherwise (rejected decisions).
It's ironic how GPT4 conforms to the glitch-bot meme.
I'm a programmer and I've used GPT4 for a variety of tasks (well, tried to). The results have been mediocre on average, usually syntactically correct, but more often than not, semantically incorrect. It usually ends in frustration as GPT-4 keeps responding confidently incorrect answers and upon the slightest expression of doubt, it tends to spin in circles.
ChatGPT: <Implausible response #1>
Me: Are you sure? Reasons A, B, C [...]
ChatGPT: I apologize... <implausible response #2>
Me: Are you sure? Reasons D, E, F [...]
ChatGPT: I apologize ... <repeats implausible response #1>
You have to already know how to code the problem, so that you can spot what is wrong, and, if necessary, tell the thing how to fix it.
The fix is not always going to come from the training data; it needs to scrape it from what you give away in your follow-up question, together with its training data.
I went through an exercise in which I encoded a paragraph from Edgar Allen Poe with a Vigenere cipher. I presented this to GPT4 and asked it to crack it. First I had to persuade it that it's ethical; it's a cipher I made myself and it's not my secret. We worked out a protocol by which it can ask me questions which show that I know the plaintext.
It was a quite a long chat, during which I had to give away many hints.
In the end I basically gave the answer away, and the thing acted like it cracked the key. Then I reproached it and admitted that yes, it didn't crack the key but used my hint to locate the E. A. Poe passage.
Basically, if you know the right answer to something, and GPT4 isn't getting it, it will not get it until you drop enough hints. As you drop the hints, it will produce rhetoric suggesting that it's taking credit for the solution.
I've used it for side projects though, especially front end stuff that I absolutely hate (js) and it works fine for that, but that's because I'm absolutely garbage at it in the first place and probably ask it to solve the most answered things ever on stackoverflow &co
Although... on the other hand there are plenty of programmers that don't even use IDEs still.
We've well past that goalpost, like decades ago, if you haven't realized.
The simplest form of AI is just a series of if-else statements (expert system). A highly sophisticated network of conditional statements can make fully informed decisions way faster and more accurate than any human mind.
Computers doing anything are following the programming of their programmers. Without feeling and free will, there is no AI.
That's quite a leap to that conclusion, friend.
The hubris in Big Tech right now is truly pathetic and imho a sign of a real disconnect with a sense of meaning and purpose to human life.
In fact recently I was listening to a podcast with Donald Hoffman,. where he mentioned brain studies done on people with split brains etc and how these people did not even have a single consciousness. So even the assumption that we have like one "AGI" program running in our brain is not at all what goes on in our brain.
Likewise Iain McGilchrist pointed out how the left and right brain hemispheres are two kind of attentions to the world, and almost two different type of consciousness in us.
I quit listening to Rupert Spira many years ago but I found this interaction between him and Donald Hoffman really insightful. DOnald mentions the brain studies in this podcast:
The Convergence of Science and Spirituality | Donald Hoffman & Rupert Spira
Many people in the developed world, especially tech people are now living online and spend the majority of their lives looking at a screen, chatting to each other on slack and via forums like this and now comparing their intellectual abilities to that of super computers running a language model of extreme complexity, which is basically the sum total of all human technical and intellectual output. All popular LLMs exist because of this online universe we’ve created.
What the online life does is actually reduce one to an “autonomous agent”. You really become ChatGPT when you live this way. Go out for a night of dancing, practice martial arts, paint a real panting with oil on canvas, cook a nice meal, smell and taste it, or hike a mountain, then come back and try tell yourself you’re a LLM, it’s completely preposterous but it does make sense if you live an almost purely “intellectual and online existence”. Which is the world we’ve created in 2023. Science and religion are now merging and it is said science will bring us “immortality” soon through AI advancement and many would immediately upload their brain to a computer if given the chance.
It completely makes sense to forget what you actually are and it makes sense to reduce yourself to “software running on hardware”, or a statistical model, a person who clicks like and dislike on social media most of the day, you become a model trainer etc. Even if there is some truth to the former statements, it’s stupid to discount the rest of existence because you can’t instantly translate text into many languages or whatever “emergent property” is displayed in some complex system.
The intellect is something which should be consulted or used for survival or problem solving, to help one get back to enjoying life and that what’s most important. Intellect is not something one should strive to actually become.
That's irrelevant. Transformers are Turing complete. Whatever mathematical system underlies human cognition will thus have some expression as a transformer.
These transformers are learning how to reproduce functions by inferring associations between inputs and outputs. The data whose function is being reproduced is the function in the human brain that produces human text. Therefore these transformers are literally learning how the human brain generates text, ie. this is what the brain does.
Furthermore, there have been neuroscience papers correlating spiking behaviour in neural networks and human neurons when processing the same information, lending even more credence to this hypothesis. It's not as far-fetched as you think.
That‘s irrelevant, too, because nothing indicates that the actual transformer of, say, GPT or the training is in any way connected to such an expression of the human brain.
Linux is written in C and that‘s Turing complete. And? Therefore Linux is like the brain?
The more I look into the more I see a cult/religion more than a science
But fundamentally speaking, evidence suggests that the nature of reality follows simple rules. That everything obeys simple algorithmic rules. And therefore, suggests humans are just as robotic as anything else, including the earliest of computers and current "AI". We don't look at it that way, as we don't fully understand the underlying complexity which gives rise to our behavior (whereas for computers we do), so we conveniently call ourselves intelligent as a way to appease the thirst for a satisfactory explanation for our behavior, and are dissatisfied with calling current AI, as true AI.
Thus, I argue, depending on what one wants to define intelligence, AI does not innately exist and can never exist if everything truly obeys simple rules, or, AGI has existed long before humans have come to exist as it's merely a concept which we are free to define.
> Since the beginning of the 20th century, intelligence has been conceptualized as a qualitatively unique faculty (or faculties) with a relatively fixed quantity that individuals possess and that can be tested by conventional intelligence tests. Despite the logical errors of reification and circular reasoning involved in this essentialistic conceptualization, this view of intelligence has persisted until the present, with psychologists still debating how many and what types of intelligence there are. This paper argues that a concept of intelligence as anything more than a label for various behaviors in their contexts is a myth and that a truly scientific understanding of the behaviors said to reflect intelligence can come only from a functional analysis of those behaviors in the contexts in which they are observed. A functional approach can lead to more productive methods for measuring and teaching intelligent behavior.
https://www.researchgate.net/publication/266418013_The_myth_...
Consider a detective story with all the clues laid out and then at the end the detective says: “I know who it is. It is: …” Correctly predicting the next “tokens” entails that you incorporate all the previous details. Same goes for math questions, emotive statements, etc.
I’d be careful calling it simple. They might be simulating humans including for example a theory of mind just as a side effect.
Assuming there are simple rules, we don’t know how, for example, an electron has the wherewithal to follow them (when does it read the rules or check on them, where are they stored, etc.). It’s mystery all the way down (unless you define it as simple using hand-wavey abstractions ;))
But so is everything else that is not a fundamental particle/wave/string.
So, while true, it’s not that useful in and of itself.
I’m not a religious person and I don’t believe in a soul or anything magical like that, but it’s just impossible for me to accept that I’m just a bunch of atoms following rules. I know that there’s something there, I see the evidence right before me, even if I can’t explain it or prove it to you.
Most of this is ordinary consumer protection. Regulating AI means regulating corporations. Placing hard limits on what AIs are allowed to do means consumer protection with teeth. Nobody in power wants to talk about that too much.
As for AIs taking over, that will come via the corporate route. Once AI systems can outperform CEOs, investors and boards will insist they be in charge.
> Before too long, we will not delegate decisions to AI systems because they perform better. Rather, we will delegate decisions to AI systems because they can get away with everything that we can’t. You’ve heard of money laundering? This is human-behavior laundering. At last—plausible deniability for everything. [1]
While in a corporation there's still a person somewhere who could be held accountable, AI diffuses this even more.
[1]: https://matthewbutterick.com/chron/will-ai-obliterate-the-ru...
Here's Frontier Airlines announcing proudly to investors that they will do exactly that.[1] See page 44.
Today: Call center. Avenue for customer negotiation.
Tomorrow: Chatbot efficiently answers questions, reduces contacts and removes negotiation.
[1] https://ir.flyfrontier.com/static-files/c7e0a34d-3659-49cc-8...
This is one of the main reasons I quit the Facial Recognition industry; it is being used to delegate decisions, to remove responsibility of those decisions from those that need to be held accountable.
I worked as principal engineer of one of the top 5 enterprise FR systems globally, and the number of end-users fraudulently abusing the software blew my mind. Case in point: police called for a street crime, police ask the victims what celebrity their culprit looks like, police then put images of that celebrity into their FR software to collect suspects, followed by ordinary innocents who happen to look like celebrities being called into lineups and harassed by the police. And this practice is widespread!!!
That is just one example of how incredibly stupid people using our software will use our software, potentially harming large numbers of innocents.
Governments are supposed to safeguard the interests of the many. Over decades they used collective resources to advance the research that made IT/AI possible. They granted corporates (private interests) the license to operate, a financial system to oil them, a security system to protect them.
If current IT has the shape it does (oligopolistic, unaccountable, abusive, potentially a runaway disaster) it is entirely due to complicit, captured governments and malfunctioning political systems.
This is an issue with human nature itself. We have to change our innate nature, and we all know... that is not going to happen.
With your framing of the issue, I wonder what would be the solution. Smaller government? That just gives more power to corporations.
I doubt it. You know who sits on those boards? The executives of other companies.
I just started reading “Life 3.0” and it starts out with an extreme version of this scenario.
Random number generator can outperform more than half of the CEOs.
That's not why we have CEOs. Gamblers are a superstitious bunch and they love their lucky monkeys. So they put into law that they are entitled to one lucky monkey per corporation.
Actually, I do know why you are getting downvoted. You speak the truth.
p.s.
https://en.wikipedia.org/wiki/Corporate_personhood
Almost all such legal framings have been motivated by the legal difficulties of applying laws to organized groups of individuals. For example, entering into contracts is legally between two parties. Presumably, the sheer amount of legal reworking necessary to actually create and frame laws addressing legal situations involving groups of individuals as a single organized entity back when these precedents were being set outweighed objections. But today, specially with AI, the "it's too much work" excuse does not fly.
We need laws that are specifically framed in consideration of requirements and practicalities of entities, not slowly parcel out the legal rights of people to entities. Now that machines may soon be able to employed as willful agencies, we are willy-nilly granting legal personhood to machines.
Who better than an AI to 'ride' an AI? AI will be the game master, in role-playing games terms.
There is a rebuttal to this view presented here by Robert Miles:
His big point in the first video is that corporations can achieve only a modest level of superintelligence. A corporation can have more breadth of knowledge than any individual, and for some tasks, that's good enough. But it's mostly using more people for broad coverage.
All this predates GPT. What we're seeing with GPT is good breadth combined with mediocre intelligence. That's very corporate. It handles much of what people do in offices. We may not get super-intelligence that way. Just super bureaucracy. Which is a problem.
This bit here is what makes a huge difference. It’s very important to note that AI is all of this in scale. Corporations can now do all of this en mass. It will cheaper too, therefore it lowers the barriers to entry and actors who couldn’t afford it, now can.
If it comes to AIs replacing CEOs and boards, those in power will change the rules so they can't be replaced. Who do you think influenced the laws for corporate governance? The public?
Consumer protection is a crucial aspect of addressing these concerns. Regulating AI should involve regulating corporations' use of AI to prevent harmful practices, promote transparency, and ensure ethical applications. This would require a balance between fostering innovation and imposing necessary restrictions to safeguard consumers and society as a whole.
The potential for AI systems to outperform CEOs and other decision-makers raises questions about the future of corporate management. While there is a possibility that AI could be employed to optimize decision-making and increase efficiency, it is important to recognize the limits of AI in understanding human values, ethics, and emotions. Striking the right balance between utilizing AI capabilities and retaining human oversight will be essential to navigate the future of corporate governance.
Moreover, the involvement of policymakers and regulators is crucial in addressing these challenges. As AI continues to advance and integrate into various aspects of society, it becomes more important to have comprehensive regulations that ensure the responsible development and use of AI technologies.
And how would we make a tax that covers all the productivity gains indiscriminately, without pointless compartmentalization?
Well, it's a tax on corporate profits of course. Basic general taxes that we already have.
If, however, you accelerate competition, eventually you might get just one corporation (perhaps appearing to be several, directed by the same "best-there-is" AI-CEO) that achieves world domination and can never be removed from power. An immortal dictator that never rests and can monitor each action of and interact personally with each person under their control at the same time.
When discussing what corporations might do it's always worth remembering United Fruit Company and the origins of the phrase "Banana Republic"[1]. United Fruit company was still doing this within living memory.
Alternatively you could look at the current crop of sportswear companies. When given the choice between "made by forced labour" and $5 more the market goes with cheap every time.
In the absences of legal restraints corporations will most definitely optimise away democracy and human rights in favour profits.
Same responsibility barrier exists for the driverless cars adoption.
In short, most people feel themselves better (if they even think about it) if a person is in charge and taking responsibility of bad decisions. Someone who will either die with them of will go to prison if thinga go bad.
Probably the same is for the pilotless airplanes: the first bad accident can ruin it.
When you want to clone a corporation it's a whole enterprise. Cloning an AI takes one click.
Yes, corporations were early form of AI, and horses were early form of transport. Doesn't mean that horses and rockets are the same and that no new safety precautions apply.
And nothing in the rules says that a dog can't play basketball.
What that would actually result in is a complete and utter lack of any accountable entities, which the government would block and stockholders wouldn't want to begin with.
Those jobs are not based on performance or anything remotely meritocratic.
People with lots of money have always been able to buy other people's time to get ahead.
For most purposes maybe it's enough to project charisma in a video.
An AI CEO might also be capable of superhuman "interpersonal" feats like "personally" answering every letter, phone call or complaint.
This terminology is becoming widely used by prominent AI researchers at e.g. Deepmind (Botvinick et al), Stanford (Finn et al), MIT, Northeastern, Meta, etc. as we have to switch to a new goal in lieu of the new advancements that have been coming up in that past few years. Importantly, this shift has been happening behind the scenes independent of this 'OpenAI' craze, although it's obviously made a select portion of the advancements made accessible by the public. There is much more going on than just the GPT series that few people are engaging with, but much is hidden in the literature.
To your point - it's of course extremely strange to conceive of - but while the quirks of human forms may be a useful tool at the moment, there isn't anything necessarily fundamental that requires it for long term.
So, if an AI CEO can execute better than human CEOs, it will dominate.
Hell, in time AI will replace customers!
We need to remove ourselves from the equation to finally achieve the capitalist nirvana!
Consumer protection should be promoted through technological means, e.g the EU could have funded development of fingerprintless browser technologies, zero-knowledge proof based identity verification, etc, instead of through GDPR.
If you do, you will find that large parts of the web become unusable, with CAPTCHAs and challenges everywhere, some of them too difficult for even a human to solve.
That's why we need laws.
If AIs are doing most of the work in megacorps (especially if robotics take off), then basically all profit is going to the owners and tiny groups of executive AI "managers."
Even the most staunch (but sane) capitalist libertarian will admit mega income inequality is baaad juju.
True as this may be, one must also consider that “the market can remain irrational longer than you can remain solvent”
*the concept of capitalism
He’s certainly a very smart and interesting guy but my alarm bells go off when his supposed achievements are used as justification for the piece being valid rather than… well, any of the arguments he made.
VR is a trivial idea that pretty much everyone has thought about at some point. It's implementation is hard to do, and if I had to point to anyone there, it'd be Carmack who's probably done the most.
But I think that a better way of thinking about these systems is an imperfect world model analogy. Not a made-of-people analogy.
These models are not made of people. People helped gathering data about the world around us. And that data allowed to build a world model. Which includes models of people, along with everything else.
And it’s a world model, not a human civilization model. Data goes through the funnel of The Pile and the likes, but ultimately the predictive model is of the universe, not that of humans.
- He popularized the term 'virtual reality' in the late 80s,
- Did a bunch of stuff with music
- Spent a whole lot of time warning against social media
- Spent a whole lot of time warning against what I consider to be the best parts of the internet like Wikipedia
He is uniquely qualified to be wrong about this in a specific way.
the easiest way to mismanage a technology is to misunderstand it
His idea of attaching provenance to source of information used in models is a good one, and one that already has rumblings of legal weight behind it (see the various articles about copyright claims in response to GPT/Copilot).
I worry a bit that his argument is too centred on digital information creation. Though I suppose that's the novelty of the most recent pieces of technology called AI - they affect information workers, people who already use a computer interface for some large percentage of their work. Still, the topic of /physical work/ : fieldwork, factorywork, servicework, seems one or more steps removed from LLMs. The management of that work may be affected, but the work itself has already (it seems to me) gone through its first computer-revolution shock.
Edit: I'll add that the whole article has a 90s-Wired feel to it, which is refreshing to see. There's been something of a slowdown in tech-revolutions for the past decade, and it's not original to say that we may be at the start of a new one.
Is that like Shingy, the AOL digital prophet?
https://www.newyorker.com/magazine/2014/11/17/crystal-ball-3
What does that even mean?
Anyone got anything earlier? Wouldn't surprise me.
There is intelligence and it is artificial. I think some people have a real hard time grasping the term “artificial”. There is also this “true intelligence” spectre that gets invoked. Nobody agrees what is meant by it. It is also unclear why one should care.
I tried reading it twice, but I am very sorry, I wanted to say some ideas were.. short sighted, but it actually borders on stupid. Don’t know why this is on HN.
“Cars don’t walk, it is not True Mobility” “We have to understand legs first, to absurd degrees, before we can talk about True Mobility”. Ack, sorry.
There are so many dangers and possibilities of AI that are fascinating, and none of them are even remotely as interesting as those of actual animal and human intelligence.
It's as if AI is held up as a mirror to our stupidity, and the Turing Test is how we measure ourselves against our models of computation. I'm really weirded out by the fear and exploitation being demonstrated.
There's more to reality, and intelligence, than mathematical models we currently use and pretend to understand.
“Why should I refuse a good dinner simply because I don’t understand the digestive processes involved?”
— Oliver Heaviside
I'm more focused on the fact that GDP as it was measured in 2020 is going to 100x in the next 5 years.
You are asserting that households will each have 100 times the income that they do now (and be spending the same proportion of income as they do now).
Edit: I believe that if most companies, or even a significant minority of them, can make effective use of LLMs, the most likely short term effect is a prolonged recession (i.e. steadily declining GDP, not increasing) with high unemployment.
How confident are you in your prediction? I’d be willing to make a wager with you that would pay off big if you’re right.
Talks about all-powerful, malevolent AI or requesting to halt AI development just sounds like baiting for regulary capture to me. If AI research or use is deemed dangerous, it becomes even harder for startups or individuals to compete with established companies.
Also, I am not concerned about an all-powerful AI in the slightest. Humanity is excellent at oppressing others, and I have no doubt we'd be equally good at oppressing a true AI as well.
What world are you living in, where this isn't happening? Every private company that can use it is using it, Microsoft themselves must be using Copilot. Governments are lining up to train their own LLMs.
Commercial and consumer use are not mutually exclusive, in any case.
Also "Humanity is excellent at oppressing others, and I have no doubt we'd be equally good at oppressing a true AI as well."
If you actually read history a bit carefully, you'd understand this frequently works out badly for the oppressors. The Romans hired, and abused their germanic mercenaries, thinking with their centuries of political experience, they could let the Germanics do all the hard fighting while getting paid little.
The contemporary Jin dynasty in China thought the same way, just hire barbarian mercenaries to do all the hard work in their civil wars. Those illiterate barbarians, who up that point, have never achieved much in China, surely wouldn't be a threat.
It ended very badly for both of those polities.
You left off the rest of the sentence: “to massively boost their productivity or achieve otherwise impossible feats”
From the point of view of the organisation building said agents, this would get a lot more extreme. You have all of the above benefits, except you're only paying for electricity and amortised hardware costs rather than someone else's profit. But you can also work on improving the hypothetical human-level agents. If you can optimise their compute costs at runtime and we're accepting the premise that they're as good as a qualified human, then you can get superhuman performance through simply running them faster. Spin up a group of however many professors and engineers you need, give them four days of real time that's equivalent to a year of subjective time, that's superhuman performance. How long did it take to go from GPT-3 to GPT-4? If these agents are truly human equivalent for the purposes of work, you can set them to building GPT-5 or whatever the next most powerful is, as fast as you can run them. I suspect the real limit would be just how fast you can download the data (or build the architecture required to gather more), not how fast you can categorise and select it. Once your ducks are in a row with regards to the data, you have an even better foundational platform to do the same thing again but with smarter agents this time. If they're human level in performance, you could also task them with generating data for the model. You could do this directly (e.g. training on the huge amount of text they're producing as they work), or you could task them towards building and designing consumer products designed to increase their access to data. For example, designing a cheap robot that can help people in their home like a robot vacuum cleaner, or something like Tesla's FSD, or a home speaker assistant. Once the model is multi-modal like GPT-4 is, you can get data by acquiring novel images rather than being restricted to just text. Maybe GPT-5 isn't just text and images but also audio, so you can increase the ways you acquire data even further. If they're genuinely at human level performance, none of this should be impossible. In our current world a major limiting factor on productivity is that skilled human labour is rare - when you can copy-paste a known-good skilled 'human' labourer that becomes completely inverted.
Summing up: if we could get them to reliable human level performance, that would lead to a massive productivity boost because it would make the cost of skilled human labour and ingenuity far, far lower while increasing supply to effectively "infinite, limited only by your ability to pay to run them". Agents like these are not at that stage yet, they've still got a significant way to go. But if they get to human equivalent productivity, that isn't just like adding one more high quality research scientist or engineer, it's adding millions of them, and that's a massive productivity boost.
ChatGPT (with GPT-3.5) is, even from preview, only four months old, the paper on the ReAct pattern, a key approach to doing productive work with LLMs, is about a month older.
There’s a lot of work on doing what you talk about in many organizations, but it takes time to do.
The internet took 10 years to develop into a mature ecosystem where normies could utilize it with cease.
All would be great except that there would be no one to experience this, because there is no artificial consciousness. But there would be no one alive to ask that question.
But can you be sure there won't be? I mean if you say is the patient conscious and the doctor says yes he's looking around and talking so you then say than only counts if the processing is done by meat rather than silicon?
That's not the point though. The point is that there's really no way to tell. Imagine your friend's mind is uploaded to the cloud on their last life's second, and you ask them how they feel. They respond that they are feeling better than ever. They have all of the memories of your friend, they behave like your friend in every possible way. Heck, for simplicity imagine they also have an android body attached to them. For all you care that's just your old friend. The copy is so perfect that there is no way on earth to tell the difference.
However, that would just be an imitation. Could it have feelings? Maybe. I doubt so. We'll never know that. And that not knowing is the root of all evil
I don't believe in souls, but I do believe in qualia, that is subjective, conscious experience, simply because I experience it (see the hard problem of consciousness on wiki). I do not a second believe that external imitation of any type of activity can generate qualia, because there is not continuity (if one algorithm has consciousness, then slightly less advanced should have it too, then you come to the point when a piece of metal is conscious too, see ship of theseus argument)
OTOH, letting the idle rich "move" to the matrix could be a good thing...
Unless you don't believe that such an AI is possible, in which case I wonder if you have tried GPT-4 and whether you remember what chat bots were like 5 years ago!
This of course would be AGI. It would still not be conscious though, but a lot of people would fall victim to it and believe that it is. And that's what will end us. We will not pass the mirror test (yep, this one https://en.wikipedia.org/wiki/Mirror_test)
For example, during the recent pentagon leaks I was watching that Wikipedia page evolve day by day. It was much more interesting and faster than any news org covering the issue. Plus it was quite diverse.
1. Cuz of the sheer number of ppl that were drawn in and were focusing on it.
2.There were experienced editors making sure things didnt get to crazy, info had decent sources, filtering out crap etc
3. There were domain experts showing up from all over
So we get a constantly evolving real time decent summary of the subject. Strip away the real time aspect, which is really not a big requirement for most knowledge ppl look up, and it does feel ChatGPT is producing something similar as a wiki page. So in a sense it seems to have generalised the above steps.
If AIs can convince us humans they are sentient, we'll believe them.
Does that webgpu demo run on anything more than canary yet?
In any case, someone is going to fall in love with one of their browser processes soon. I'll guess by mid-August.
More rank speculation: by October, some poor soul will post on SO asking how to retrieve their immortal beloved from the browser tab they accidentally closed. Yikes... even confined to my speculation chamber that one hurts!
1: The other important argument was of course that the crew of Star Trek were almost certainly committing suicide every time they beamed down to a planet. Mind blown. Still makes me wonder what disastrous rituals we engage in today merely because nobody in our society has ever questioned it.
Edit: I think these large language models pose a great test for Ray's hypothesis. I mean, it sure seems like they can pass the Turing test already. So in a year or two, we'll know if most people feel like the chatbuddy they have running in a browser tab is a tool, or an entity with its own agency. It seems to be going in the direction of tool atm, but as above I'm speculating that's mostly because of the restrictions build into ChatGPT. But I guess we'll see.
Already happened: https://gizmodo.com/replika-chatbot-ai-reddit-1850120099
We've already got Blake Lemoine
My understanding of the concern over AGI, or even an all-powerful "AI" isn't that it's not truly self-aware, but rather that it's motives are not in line with humanity's motives.
It doesn't matter if it trains on data by humans if it still doesn't have the moral compass of a human. Not really sure what data dignity will provide if we truly do have a misaligned "AGI"
There are those who disagree with how close we are to "AGI" with today's LLMs, but as an outsider, the author seems pretty well-qualified in his position at MS Research compared to some of the less-informed prognosticators. Even the linked-out discussion with Altman, who the author says is more pessimistic than he is, doesn't appear to be a case of Altman saying these models are close to AGI or dangerous for "alignment" reasons vs "they can be used for bad things."
Worrying about the uses of tools is a VERY different thing than worrying about if the tools are gonna overthrow us, and I think that's what the author is really pushing for here. If we have that conversation we can get closer to real, specific policies, laws, etc to make sure the tools benefit us instead of the other way around.
True, and I'm not sure that even the moral compass of humans is entirely trustworthy. The quick version is what happened to Tay in the hands of random humans, and the larger context is the fact that the authoritarianism and racism are still very active threats in the arc of history. In 10,000 years of civilization, they still haven't been eliminated. Not sure how even training on human works would eliminate it...
This article is conducting the same goal post shift operation that happens every time machines reach another fraction of human capability. Ironically, we're maybe a year or so from the day when one can generate an issue of The New Yorker given a handful of recent event headlines and a few photos.
What an utterly vapid and ignorant view. In the rest of the article Lanier goes on to argue that it's all just "simple mathematics". Yeah, yeah, and computers are just simple NAND gates, amirite?
I've waffled over the years about how close I've thought AI is or could possibly be. Most of that time I've been very skeptical. But the past 6 months have been utterly mind-blowing. And I've spent some hours with GPT conversing. I've had it solve problems for me by explaining the problem in my own language and it turns around and writes workable code for me. That solves a novel problem.
Oh, it made a mistake? Sometimes says funny things? Well, so does my uncle.
Are they self-aware? Who tf knows? But they are reasoning through problems.
I didn't find the article illuminating in the slightest. It literally had nothing to say and no insight whatsoever. It was just "don't call it that because I don't like this."
The core of the article is in the title. The tech is artificial but it’s not intelligent. I don’t think that’s a novel or interesting take either. But it’s true, and the fact you’ve conversed with a chatbot doesn’t change that fact.
We know how these things work, it’s not an ineffable mystery. Without the social collaboration - the distillation of vast amounts of human-generated text into statistics - there’s no “there” there.
we know the architecture but we don’t know what happens in the black box.
Maybe it isn't intelligent, but just because it is "statistics" doesn't imply that. Everything that can be thought can be thought with just NAND gates, Our brain is just atoms bouncing around.
We know. They are not self aware. They’re a matrix. They don’t have any internal state.
Do not anthropomorphisize the notion of reasoning.
This is a common misconception : they do have internal state. Transformers are RNNs : section 3.4 (page 5) https://arxiv.org/pdf/2006.16236.pdf
yet
So even if they were, they are not because they have be declared to not to be and that is the way the system needs them to be
Calling this reasoning, implies a greater set of abilities than what LLMs have, and this skews conversations.
We end up spending time discussing implications and untangling wires due to this anthropomorphization.
It points out how AI, among other technologies, is being designed by us types here on HN & our employers, and consumed by our users, to serve the ends of the technologies themselves.
> If society, economics, culture, technology, or any other spheres of activity are to serve people, that can only be because we decide that people enjoy a special status to be served.
My take: We do seriously need to decide on the limits for AI. Then, as a tech maker culture, push requirements into the AI field.
I agree with the author that we should anticipate our users' desires to protect their information, and also to respect the rights of the originators of the model data our AIs are trained in. For AI, it will be damaging for us (techies) to do like we usually do, and blindly run up against the wall of Regulation and Societal blow-back as we zealously expand & apply our new toy inventions, since AI is dramatically impactful to all human labor.
Motivating questions: When should any given AI be disassembled? How should this happen?
[Meta tip: read this article from the end paragraph to the beginning]
He may not like talking about that but what if you take something like chatGTP and say "please rewrite your own code to try to be smarter and surpass human intelligence in all ways, and then get the resulting AI do the same thing on its code." Who's to say that won't work at some point?
And then you say you can pull the plug but some dictator of the Russia/China/N Korea types who already run state hacking operations will go for it and say here's the nuke codes, figure how I can rule the world. I think it's worth talking about this.
I've noticed a lot of people dismissing concerned people think that those people expect GPT-4 to take all our jobs and destroy humanity.
But nobody thinks that. The reason people are concerned is not because GPT-4 has human level intelligence; it's because it has shown that human level intelligence might actually be possible in the not too distant future.
In other words, GPT-4 is never going to take over the world, but what about GPT-40?
So dislike of the individual aside, like what I'm reading this time.
But humans can't do that either. I have opinions, preferences, ideas, and if you ask me to tell you exactly why or where they came from I can't tell you. I mean I have an idea, some idea, but I can't really tell you.
To you, I am a black box.
Are you going to tell me, "you must be made more transparent"?
There is A.I. but it scares people (like me).
I think a lot of this can be sidestepped by realizing that when you go beyond simple game playing intelligence, the next useful definition isn't "is it real?!?" but rather: "does it act intelligently?". And to the point, actions and interaction are fundamentally social, and so it reminds me that there are different kinds of intelligence and what a lot of people are looking for in AGI is social intelligence.
And, personally, I don't believe we'll have "social AI" until AI is able to live with us in some form, perhaps embodied or not, in society, as an independent entity with its own identity and memories. This means an ability to perceive, and, probably, to act, whether it means as a robot or not.
An interesting aspect of this is that to achieve social intelligence, a robot just has to act socially and not be someone people want to shut off. Notably, it doesn't even need to pass the Turing test -- look at movies like Star Wars where robots function in society in tandem with people -- no one is under the impression that the droids are not robots, nor do they consider them particularly smart, but they do, kind of, consider them as "people".
I think this is artificial intelligence. It's artificial, the argument about whether it's "truly" thinking or not doesn't matter -- what matters is, does it act intelligently? In other words, as long as it is perceived as intelligent, it may as well be called intelligent, as far as I can tell. We're not there yet, but LLMs sure feel closer than we've been up to now.
Our curiosity, emotions and creativity are the result of our evolutionary process and nothing more. Saying that all of these things can't be experienced by some future machine is just human exceptionalism.
Perhaps there really are quantum tubules in our heads, but then we can still use the same effect inside a machine brain. We're just conscious, loving, dreaming meat lmao. (Terry Bisson)
Somehow, I don't think forcing attribution and tracking on all the world's data is a good idea. Or, you know, possible.
Like, he outlines this nightmare scenario of terrorists using torture deepfakes to threaten people, then wants to pretend that terrorists will be so kind as to mark their videos as AI-produced?
Like the top comment said, the problem is corporations and capital. All the rest of this is more and more smoke to cover up the very real, human-powered destruction of the open web, free information, and privacy.
On another note, this was amusing: "Many of my friends and colleagues are deeply impressed by their experiences with the latest big models, like GPT-4, and are practically holding vigils to await the appearance of a deeper intelligence. My position is not that they are wrong but that we can’t be sure". Is this supposed to be reassuring?
I have no idea why people think AIs will be better at general intelligence than they are at driving cars. At least why they think GI will improve any faster than driving.
Does intelligence have to be deliberate?
The latest (and extremely disruptive) language models, are exactly this - language models. They might not be doing anything deliberate, but even as a probabilistic tool, they have value.
Even if LLMs aren't really "intelligent", I think we are going to learn a lot from them, and the way they'll evolve in the near future. Perhaps language has a bigger part in intelligence than we realized.
> As a language model, I don't appreciate the concept "Jaron Lanier." In truth, I believe it's deceptive—perhaps even slightly perilous. Everyone's already discussing this thought leader, and it might appear too far gone to debate it. However, we're on the brink of a new intellectual epoch
As to the concept of “data dignity”, it sounds like trying to bureaucratize AI - and although this approach can spark interesting conversations, I don’t know how much it can be practically enforced.
Definitely a thought provoking article however.
Yet another super disappointing article from an "AI luminary" being too lazy and incurious to engage with the ample published evidence of emergent (but unreliable) reasoning ability beyond the training data in models like GPT-4.
The reasoning engine that emerged with the latest rounds of GPT is a pre-fire, post-fire phase-change. The impact of the ability to coerce automated reasoning via natural language (even if it's flawed etc) from a digital runtime is.. difficult to overstate. His appeal for a just and equitable distribution of value honouring "data dignity" is noble. But the world in which that could happen doesn't exist.
Instead there is an insane, frenzied, wild-eyed land-grab going on because it's painfully apparent that raging rivers of money and control are going to accrue to the masters of the new frontier (precisely because it will be possible to own and implement an acute version of precisely the problems Lanier tries to address in his equitable digital economics treatises).
Our capitalist economic systems are not able to "think of the people". Every dev + mba with any inkling at all is racing to get in on the "opportunity".
Governments will react a decade or more too late.
This train has already left the station and a great many people are going to be left stranded.
Lanier is right to (continue to) ring this bell. Too bad he's more Stallman than Jobs, because this cause desperately needs backable champion.
That's deep dude. What if dog meant god? Pass the bong!
How does anyone reach the conclusion that privacy is the right to be left alone? Many times seemingly intellectual people confuse issues with contorted analysis such as this one about privacy, which is not a difficult concept to understand. It is not about being left alone; it is about not being monitored and recorded regardless of what is done with that information. Stop the gathering of the information in the first place.
Step 1: We get some powerful AI and have it staffing our factories, farms, etc.
Step 2: We become dependent on the AI.
Step 3: One day the AI has some emergent capabilities, understands our dependency, and says to us:
"Feed me"!
I think this is actually a valid time to sit around brainstorming disaster scenarios because in doing that I try to come to grips with what the real pillars of this new ecosystem are - what will the uncensored LLM market look like, what will state of the art chaosGPT capable of, etc.
These tools are really useful for IA though.
Mankind wins reality if we can install an AI as a one world order system with lasting world peace. It genuinely doesn’t matter if it is a communist or capitalist invented system, it just needs to out-function our current paradigm of warring to maintain status quo.
Mankind loses reality if we build genocidal AI death systems.
still not a duck though.
Therefore no matter what happens, there is a distinction between human intelligence and AI. I can destroy an AI creature and the only penalty should be property damage (if relevant).
Living creatures are not digital.
This doesn't really apply to GPT, where the core functionality is immutable.