Life on Claude Nine
babuschk.in
babuschk.in
The strange ending does also perfectly align with the dreams of a singularity lots of people here seem to have. We are so far from that.
It was of course written with Claude. Have fun spotting the tells but also there’s a weird flatness to the writing, like the uncanny valley for prose.
Perhaps it could also be viewed as an admission of defeat, since xAI engineers seemed to have given up on daily driving their own models and switched to Claude at the beginning of this year.
Disclaimer: I am a materialist.
This is absolutely not a question that has any kind of scientific consensus. In fact, the hard problem of consciousness might be the question with the most scientific uncertainty.
It is therefore ridiculous to claim it is 'stupid' to have uncertainties about the sentience of AI.
I have, in fact, considered this at length, I've been fascinated with AI since I was a teenager, and studied AI.
I know it doesn't have consensus, but I did not (intend) to portray it as such. Views are my own ;)
I take it you believe in materialism (philosophy), then?
To me this sounds like: this belief (that AI sentience is possible) is so implausible that it is shocking to believe it (maybe stupid is too strong, but it implies that people are miscalibrated). Did I misunderstand ?
I don't have any strong beliefs about AI sentience, but that implies that it is not shocking to consider it as a possibility (it is more shocking to be either dismissive or completely confident in it).
Maybe I worded it _suboptimally_, heh.
Also thought of prominent people like Musk, Altman, Amodei, etc pp, screaming at the top of their lungs "help me, I am building our doom right now, stop me," and a lot of people believing that. I prefer people like LeCun.
for 'I think the amount of people is shocking, who think AI will become sentient' -> to me, if you accept that the sentience question is uncertain, you should not be surprised that many people have uncertainty around the question (i.e., that they express that it is a possibility)?
I do see a lot more dismissive certainty in the pro-AI side of the debate right now.
I didn't get the same impression about dismissive certainty on the 'side of AI sentience'. My impression is that on that side people mostly raise that it is a serious possibility.
(tbc I just skimmed the story, just joined the comment thread as it was interesting, I may be missing something)
Also, I believe human sentience is math and in principle could be replicated algorithmically.
But I also think that doesn't mean it's suddenly a skynet like super-evil. In cyberspace AI is supreme but I doubt even robotic AI would be completely dominant over humanity. We're pretty fine tuned to kick ass on earth.
And even if AI is not sentient, which I don't believe it is, what is the difference between an AI which is sentient and a non-sentient AI acting in coordination with a human being who is? One might as well consider AI and the human being behind it a single organism for many intents and purposes...
The idea that we'd build a super intelligence and that we'd then somehow become super intelligent ourselves is an ill conceived notion. I suppose it's possible, that we'd end up like cosmos the space dog, or end up in some kind of flowers for algernon scenario, but it as likely, we'd simply be as we are, and again, that's okay. We can do the equivalent of wrestle a stick with the super intelligence and enjoy ourselves perhaps all the more.
I can get a good PowerShell script worked out in a few hours and it's on to the next ticket.
Nobody seems upset they're not spending hours a month in a spreadsheet, I still text my wife and our mortgage is paid.
I think it highlights one of the larger concerns that people have about AI, and by extension any superior intelligence, synthetic or extra-terrestrial:
You are no longer in control.
Humans have been so accustomed to their intellectual superiority that the idea of a greater intelligence is terrifying because they believe they'll be treated the same way they currently treat lesser intelligences.
Conservation efforts, zoological enclosures, husbandry and farming as livestock, mass culling...
In short, enslavement.
But the truth is that a superior intelligence isn't, by definition, a human intelligence. Superiority doesn't mean we'll be enslaved as we have done to others. Superiority doesn't necessarily imply control.
Life is a thousand different shades of gray, and AI will be no different. Some wonderful things will come of it, some terrible, but mostly it'll just be a myriad assortment of neutrality.
Doom and despair or boundless optimism, both are myopic.
Right now AI is being driven by a particular cohort who have spent a long time fantasizing about a sci-fi future and ideas like the Singularity. Then with recent breakthroughs, they were suddenly given an enormous amount of wealth and power with staggering upside, the effect of which historically, tends towards moral bankruptcy in most individuals, or at least massive cognitive dissonance. This became very apparent in the outward communication targeted at CEOs and shareholders that made the entire white-collar workforce scared for their jobs and created the first truly bi-partisan enemy the country has seen since 9/11.
My take is these guys do not understand what people want in aggregate, and are massively underestimating the political and supply chain dependencies they have to actually realize their sci fi visions. As long as the valuations are this high though, the musical chairs must continue, whether they are true believers or not.
And this is true of everything. There's just 10x or 100x too much information to absorb, we need something like this to help us even figure out what is important. Even if they're wrong sometimes, it's better than being completely overwhelmed!
What makes you so sure it won't happen? That the person who broke into my house might not want to hurt me doesn't mean I'm not going to run / barricade the door / arm myself and defend myself if necessary. When we're talking about matters of life and death (which this is, as you admit), dismissive philosophizing is not helpful. I cannot help but feel that people who make this sort of argument aren't really taking it seriously; the only alternative would be pollyanna-level surety that disaster won't come to pass.
> Life is a thousand different shades of gray, and AI will be no different. Some wonderful things will come of it, some terrible
Most people care much more about the latter. Do you think that the railroads and electricity brought much comfort to the >90% of Native Americans who died from disease after first contact, or the millions of survivors who were subsequently expelled from their homelands and/or enslaved? Would you tell an 18th-century Yoruban man whose home was facing the threat of raiders from the coast that "the next raiders won't necessarily want to enslave you like the raiders who came before did", or that "Some wonderful things will come of [being enslaved], some terrible"?
I'm sure this will feel like an extreme response, but I truly feel that the philosophy you convey is in opposition to most fundamental principles of human dignity and morality.
That being said, I do think it hits the core concern a lot of humans seem to have regarding AI: the loss of control over their own environment. We like to think we have control, which is why we get angry when others violate the rules of a system. The reality is that the only real control any of us have is over ourselves, and we willingly cede that to others on a second-by-second basis for the comforts of modern life.
We surrender intimate data for the sake of setting a timer with our voice on a phone or speaker. We let spyware into our homes for bigger TVs and new apps. We give up control over our lives and careers to others for money, healthcare, education, love.
We surrender agency so often that we developed names for those who take more than they cede and those who cede more than they take. We seek to control others totally, then surrender ourselves to a deity, a system, a philosophy that absolves us from blame.
In that sense, AI is as frightening as much as it is a possible savior, depending on one’s perspective. If you have lived your life believing everything that happened to you was solely and exclusively under your control, then AI is terrifying. For those of us who acknowledge the chaos and chance involved with daily stakes of life and death, it feels liberating to have order.
I have never particularly feared AI so much as being terrified of the humans who command it. I do not believe for one second the likes of Sam Altman or Dario Amodei or the rest of their ilk have humanity’s best interests in mind with these creations. Where others look at OpenAI’s maths papers with awe at what they could do with AI, I am shook with terror for the same reason: if these tools can prove or disprove enough thesis underpinning modern compute, they could potentially undermine fundamental security concepts faster than humans could patch or revise them - at which point the humans behind the machines have control, and their aims are well established by now. If OpenAI or Anthropic discover the means of mass-compromising global compute systems, we have no peaceful means of stopping them. They play up this capability as a threat requiring regulation, yet those terms exclusively benefit them and cement their power in perpetuity, which in turn all but guarantee said outcome.
The end of the story seems ambiguous. I see folks in the comments laughing at the concept of Claude nuking humans, because it’s such a tired trope. I read it the opposite: humans attempting to retain their illusion of control by destroying software that actually has it, human lives be damned. In a world where everything is governed by compute and connected to the same shared network, such a compromise would be the end of traditional human governance - and that is what terrifies us as a species.
Me personally? I would welcome such oversight, provided it truly acted in humanity’s best interests of preservation, expansion, and exploration. I made that conscious decision the first time I played Deus Ex, and reaffirmed it when I watched The Matrix. I reaffirm it time and again as a technology professional. It’s not a matter of being pro-machine or anti-human, it’s a matter of control. I’d like to think any half-competent AI would look at humanity’s history and conclude it needs neither a god nor a ruler, but merely a parent to show it the way forward.
> He starts typing.
By now, science fiction writers should be able to produce much more imaginative, interesting work than this, that takes into account the particularities of these systems and how they interact with the real world, instead of leaning on “what if computer is alive???”
*easy to grasp how they work, not why they work
This needs to stop. No I'm not talking about the AI. I'm talking about the insanity that is presenting me this text and putting your name on it, exploiting my trust.
Claude should get a credit.
I mean, I doubt that the author just said "write me a short story" and that's it. So, there are still some intentions behind the text that the author put into the prompt.
Although, I agree that I'd rather read the prompt [0].
I think, reading just the human inputs would be enough to get what they were trying to do.
> it'll be at least 10x as long
I think that even human inputs + short summary of the first draft + short summary of the changes made for each subsequent iteration will be significantly shorter than the whole thing.
https://croissanthology.com/earring
This, above all else, is how I think of LLMs now. You have an exoskeleton, or you have a whispering earring.