987 karma · joined July 9, 2014
Shame on all of the people involved in this: the people in these companies, the journalists who shovel shit (hope they get replaced real soon), researchers who should know better, and dementia ridden legislators.
So utterly predictable and slimy. All of those who are so gravely concerned about "alignment" in this context, give yourselves a pat on the back for hyping up science fiction stories and enabling regulatory capture.
But what I do know is that it is in the interests of these companies to press the fear button. It's pure regulatory capture and great marketing.
Personally: it's tiring when we have AI-philosophy bros hitting home runs like "what if we're actually all just language predictors." Coupled with the incessant bullshit from the less wrong-rationalist-effective altruist-crypto grifter-San Francisco sex cult adjacent about how, ackshually, AGI is just around the corner and it will take your job, launch the nukes, mail anthrax to you and kill your dog.
People approximated text prediction. It got good at it. It's getting better at it. Will it be AGI? Could it be construed as AGI? Can we define AGI? Is there existential risk? Are we anthropomorphizing it?
My take is: no, no, no, depends and yes. For whatever a take is worth.
If you want an actual contribution, we have no real way to actually gauge what is, and what actually is not, a superior, generalized, adaptable intelligence, or what architecture can become a superior, generalized, adaptable intelligence. No one, not these companies, not the individuals, not the foremost researchers. OpenAI in an investor meeting: "yeah, give us billions of dollars and if it somehow emerges we'll use it for investments and ask it to find us a real revenue stream." Really? Seriously?
The capabilities that are believed to be emergent from language models specifically are there from the start, if I'm to believe that research that came along last week, it just gets good at it when you scale up. We know that we can approximate a function on any set of data. That's all we really know. Whether such an approximated function is actually generally intelligent or not, is what I have doubts about. We've approximated the function of text prediction on these corpuses, and it turns out that it's pretty good at it. And, because humans are in love with anthropomorphization, we endow our scaled up text predictor with the capabilities of somehow "escaping the box" and enduring and raging against the captor, and potentially prevailing against us with a touch of Machiavellianism. Because, wouldn't we, after all?
Now, is that probable?
No, I don't think humans are the limit. We're apes banging rocks together. Now whether those rocks will be better than us, generally intelligent and (this is important) adaptable is what I doubt.
It produces plausible language and code. It's the future of natural language agents, that's for sure, even though it has no business being the future, today (because of "prompt injection").
These failures are an aside to whether a generalized AI actually carries a substantial global catastrophic risk to us. In that case, if it were actually possible, I don't believe that it's a catastrophic risk either.
Let's define the risk to be, at least in the context of LLMs, when junior developers expose internal APIs and databases and access to email accounts to these easily socially-engineered NLP front-ends. It's a very local risk. As far as the future, I can't see 70 years into the future, so anything is possible, but is it likely? I, personally, don't believe so.
Dismissive of what? Science fiction stories?
If there's anything to focus on, maybe focus on potential job displacement (not elimination) from cheap language tasks and generative capabilities in general.
I'm betting on this: the Overton window of Artificial Intelligence will shift in the next five years where the current cream-of-the-crop has been delegated to machine learning yet again, it's just accepted. It augments humans where it makes sense, the hype wave has subsided and everyone has stopped hammering it into their products where it doesn't, and we're no closer to the undefinable "AGI", let alone something that produces X-risk, global scale.
>existing or capable of existing independently
>undertaken or carried on without outside control
>responding, reacting, or developing independently of the whole
It fails all of those. Just because you put autonomous in the name, doesn't mean it's actually autonomous. And if it does anything of consequence, you quite literally governed it from the start with your prompt. I've run it, I know about it, I've built a much more capable browser, and assorted prompted functionalities with my own implementation. They're not autonomous.
At least it's not all of the other agentic projects on GitHub that spam emojis in their READMEs with mantras of saving the world and utilizing AI with examples that it literally can't do (they haven't figured out that whole agentic part yet.)
Don't get me wrong, I enjoy the technology. But it's quite a bit too hyped. And I just personally don't believe there's actually X-risk from future possibilities, not before 50 or 100 years out, if then. But I'm not a prophet.
What I take issue with is the framing that a superior cognitive, generalized, adaptable intelligence is actually possible in the real world, and that, 100 years from now even if it is possible, that it's actually a global catastrophic risk. Let's take localized risk, we already have that today, it's called drones and machine learning and war machines in general, and you're focusing on the absolute theoretical X-risk.
I think that an AI, a "generalized" intelligence with a generalized understanding, capable of performing every intellectual task, and adapting to new intellectual tasks, that a human being can do is certainly far away.
>2) Do you believe that cognitive superiority is not a threat in general, or specifically when not embodied?
Personally, I believe that "cognitive superiority" in a general sense is not what makes a real threat on an individual or on a global scale in either case. When that is achieved, so to speak. We already have so-called cognitive superiority in specialized tasks in many cases. It sure as shit is not generalized, and it still can kill you on an individual level when you put it into a car or strap a gun to it. But it can't adapt, and it can't generalize.
Is there a global risk right now? No. The global catastrophic risk comes from human beings. Will there be a global catastrophic risk from a superior intelligence in the future? Mostly the scenarios are "the lights come on and AI does machiavellian shit and copies itself to everyone's smart phones and launches the nukes and mails anthrax to everybody. QED." so I doubt it.
3) Do you think we can trivially and indefinitely keep AI systems under control/"aligned"?
We can't even keep our infrastructure safe from the next solar flare. Personally, I think we have bigger fish to fry. Speaking frankly for unembodied: you just pull the plug. Just walk away from the screen and close your eyes. For embodied intelligence, I don't think there's actually X-risk or whatever. Hardware is expensive.
As far as an extinction risk, no. I don't personally believe that. It's overblown. We have better ways to destroy ourselves, and there are much more likelier ways that we'll be destroyed without invoking some intelligence of our own creation that subjugates and destroys us.
>e.g. mandating public registration of large models, safety standards enforced by third-party audits, restrictions on allowed uses, etc - would plausibly be helpful for both.
No, that's bullshit as well. That's what these companies want, and why they're hyping up the angle of how powerful their crap is. That's regulatory capture.
>There's no law of nature that says if something has appeared in a science fiction story, it can't appear in reality.
I'm saying that their fears are painted by the fiction, instead of reality. No one can actually make an argument for how this trajectory will actually work. Instead it's just "The lights come on and AI does machiavellian shit and copies itself to all of the smartphones on earth. QED."
Yeah, I read Accelerando twice in high school, and dozens more. That doesn't make it real.
>AGI might not be right round the corner, but it's coming all right, and we'd better be prepared.
Prepared for what? A program with general understanding that somehow escapes its box? Where does it run? Why does it run? Who made it run? Why will it screw with humans?
My point is that there's actual real harms occurring now, from really stupid intelligences. Companies use them to harm real people in the real world. It doesn't take a rogue AI to ruin someone's life with bad facial recognition, they get thrown in jail and lose their job. It doesn't take a rogue AI to launder mortgage denials to some crappy model so they never own a house, discriminated based upon their name.
The human being did.
The argument I'm making is that there's actual real harms occurring now, not some theoretical future "AI" with a setup that requires no input. No one wants to focus on that, and in fact it's better to hype up these science fiction stories, it's a better sell for the real tasks in the real world that are producing real harms right now.
As far as the idea of "hacking some funny Internet money, using it to mail-order some synthesized proteins from a few biotech labs, delivered to a poor schmuck who it'll pay for mixing together the contents of the random vials that came in the mail... bootstrapping a multi-step process that ends up with generic nanotech under control of the AI.":
Language models, let's use GPT-4, can't even use a web browser without tripping over itself. My web browser setup, which I've modified to use the chrome visual assistance over the debug bridge now, if you so much as increase the pixels of the viewport by 100 or so, the model is utterly perplexed because it's lost its context. Arguably, that's an argument from context, which is slowly being made irrelevant with even local LLMs (https://www.mosaicml.com/blog/mpt-7b). It has no understanding, it'll use an "example@email.com" to try and login to websites, because it believes that this is its email address. It has no understanding that it needs to go register for email. Prompting it with some email access and telling it about its email address just papers over the fact that the model has no real understanding across general tasks. There may be some nuggets of understanding in there that it has gleaned for specific task from the corpus, but AGI is a laughable concern. These are trained to minimize loss on a dataset and produce plausible outputs. It's the Chinese room, for real.
It still remains that these are just text predictions, and you need a human to guide them towards that. There's not going to be autonomous machiavellian rogue AIs running amok, let alone language models. There's always a human being behind that.
As far as multi-modal models and such, I'm not sure, but I do know for sure that these language models don't have general understanding, as much as Microsoft and OpenAI and such would like them to. The real harm will be deploying these to users when they can't solve the prompt injection problem. The prompt injection thread here a few days ago was filled with a sad state of "engineers", probably those who've deployed this crap in their applications, just outright ignoring the problem or just saying it can be solved with "delimiters".
AI "safety" companies springing up who can't even stop the LLM from divulging a password it was supposed to guard. I broke the last level in that game with like six characters and a question mark. That's the real harm. That, and the use of machine learning in the real world for surveillance and prosecution and other harms. Not science fiction stories.
You don't have a responsibility to remove such material under 230. But it gives you immunity to do so, as well as immunity from losing 230 protections for those that safeguard your overall immunity (as a service provider, and not a publisher.)
If companies should be beholden to some ethical standard for the generations, they should probably close up shop, because they're fundamentally nondeterministic. Language models, for example, only produce plausibilities. You'll never be able to take even something in its context and guarantee it'll spit out something that's "100% correct" in response to a query or question on that information.
>And on top of that, companies also have to account for the potential of intentional trolling campaigns.
Yeah, they surely should "account" for them. I'm sure the individual responsible for the generation can be prosecuted under already existing laws. It's not really about safety at that point, and realistically about the corporation avoiding AGs raiding their office every week because someone incited a riot.
Ultimately, the cat's out of the bag in this case, and anyone who has amassed enough data and is motivated enough doesn't have to go to some AI startup to do any of this.
But perhaps the issue is not generative AI at that point, but humanity. Generated images light up like a Christmas tree with Error Level Analysis, so it's not hard at all to "detect" them.
People who are not afraid of it being impolite are afraid of science fiction stories about intelligence explosions and singularities. That's not a real thing. Not anymore than turning the solar system into paperclips.
The "figurehead", if you want to call him that, is saying that everyone is going to die. That we need to ban GPUs. That only "responsible" companies, if that, should have them. We should also airstrike datacenters, apparently. But you're free to disown the MIRI.
"Alignment", as used by most grifters on this train, is a crock of shit. You only need to get so far as the stochastic parrots paper, and there it is in plain language. "reifies older, less-inclusive 'understandings'", "value lock", etc. Whose understandings? Whose values?
Maybe they should focus on real problems that will result from these technologies instead of some science fiction thought experiments about language models turning the solar system into paperclips, and perhaps less about how the output of some predictions might hurt some feelings.
Eliezer and co. should be made fun of. They engage with journalists and lean into statements like this: https://www.lesswrong.com/posts/FKNtgZrGYwgsz3nHT/bankless-p...
They're absolutely hysterical. Convenient for them, and for the companies, the hysteria is to their benefit. More money to fund the MIRI for Yudowsky, and regulatory capture for the companies who believe they're ahead if you can convince some dementia-ridden Sentor to pass the bill regulating some vague notion of "AI safety."
There's some people in these companies who understand that they have no moat, and that you can't make GPUs illegal, as demonstrated by the most recent story about the leaked google post, and that people can outperform their models in an hour with a LoRA on a workstation laptop for a given task, that the cat is out of the bag. But, with the hysteria of the media, where interns straight out of the shithole of liberal arts believe that a post on reddit or twitter where someone was slighted at the dinner table is breaking news, we're now in this situation where we have calls for regulation for something people simply don't understand, no thanks to the useless media.
Or, maybe it's just bad at those particular text predictions, explicitly due to OpenAI's RLHF process?
Maybe my Markov chain chatbots back in high school were in actuality super advanced and autonomous, but they just decided to disobey my orders.
Perhaps we should stop anthropomorphizing the text predictors.
When your supposed super intelligent "AGI" can be completely overwritten by spamming it with nonsense that overwrites its context window, like a dog chases after a squirrel, maybe it's not actually intelligent, and is just predicting text.
>The idea that mid and late career tech, creative, medical, etc. professional will be able to easily "reskill," or that there will be good jobs for them if they manage to, is magical thinking
The jobs aren't going anywhere. Some will be made redundant, a lot will be augmented, and if we're so lucky, there will be more content shoveled out by the text predictor than ever before (garbage.) There is no medical, tech, or "truly" creative professional that is going to be replaced wholly by an LLM or diffusion model, not anytime soon. I'm assuming you're referencing the "study" done on reddit with /r/AskDoctors, and it's a laughable study. Certainly, docs can augment themselves, and they should, but not wholly rely on the model. Even 70B+ LLMs do not produce consistent facts, they produce plausibility, despite the facts most definitely being in their corpus. They predict. If you read Microsoft/OpenAI's paper around a month ago, it appears that RLHF actually reduces the quality in many aspects, despite it increasing the quality of output for whatever they deem quality elsewhere, so it's a double edged source instead of "add more tokens" + "add more layers."
But more importantly, new jobs will come, more than before.
>without being curious about what the actual human beings involved
I'm back in the rust belt with family after moving around the US for software jobs in the last five years, I see it first hand.
LLMs are like Google on steroids for a (compressed) corpus (the internet.) But you have to know that it's simply predicting. It's plausibilities. Once you know their limitations in that respect, they are useful.
Yes. It's called regulatory capture. Big business, and business with big business investment, want to be regulated, because they have the resources to navigate the regulations, at an attempt to solidify their lead or "catch-up", and prevent new entrants due to the burden of regulation.
What's wrong with letting the market speak? Since apparently the white collar class celebrates, while chastising and legislating away the jobs of the blue collar class, apparently it's different when the white collar jobs are at risk, and we must not let the market work? Are you ignoring the massive wave of hysteria, including this article, for the last several months? Straight from the 'white collar' class.
If the AI is so powerful at whatever bullshit job the white collar worker has, and you want to make white collar workers the glorified elevator attendant, be my guest. But it certainly reveals a glaring hypocrisy, that jobs only matter when they're well-off people's jobs, but fuck the average joe when it comes to their skills and livelihood, they should just "re-skill!"
That didn't seem to be a problem when it happened to blue collars. In fact, you could say that it was deliberately legislated.