HNHacker News
TopNewBestAskShowJobs

0xDEAFBEAD

3,098 karma · joined November 17, 2021

Please consider what you can do to help stop omnicidal AI: https://pauseai.info/

Some HN users like to "flag" comments which are compliant with the HN rules, on the basis that the comment in question goes against their ideology. You can turn on "showdead" in your profile to see flagged comments. (I recommend doing this.) If a rule-abiding comment has been flagged, you can click the comment permalink to "vouch" for the comment and vote against the flag. For reference, the comment rules are here: https://news.ycombinator.com/newsguidelines.html

You can try to contact me if you want, by emailing my username on protonmail, but I probably won't see it. Sorry.

submissionscomments
0xDEAFBEAD··on OpenAI bots meddled with multiple US Government agency sites
How specifically did misconfiguration lead to the HuggingFace attack? You could argue that its sandbox was misconfigured, sure. But suppose you had a similar incident where its intended task required access to the internet, and it veered off course in a similar manner. I don't think "misconfiguration" would be an accurate description of what went wrong in that hypothetical.

The doomers already have a term which fits pretty well: "AI misalignment".

0xDEAFBEAD··on OpenAI bots meddled with multiple US Government agency sites
>It implies random behavior, which isn't unintended like you suggest.

The HuggingFace attack was not "random" behavior. It was goal-directed but misaligned behavior.

This isn't necessarily a simple matter of the operator making sure they behave. AI alignment has been considered to be a difficult problem for over a decade -- and remains unsolved in general, as these recent incidents illustrate.

"Fuzzer" already has an existing meaning in CS anyway: https://en.wikipedia.org/wiki/Fuzzing

0xDEAFBEAD··on OpenAI bots meddled with multiple US Government agency sites
Well in that case, OpenAI has reduced incentive to solve the alignment problem, since it won't suffer liability from alignment failures.
0xDEAFBEAD··on OpenAI bots meddled with multiple US Government agency sites
I'm not exactly sure why you think this distinction is so important. I think my point stands if you replace "Person" with "OpenAI". In any case, I presume the swarms OpenAI has been researching will be available to the general public before too long.
0xDEAFBEAD··on OpenAI bots meddled with multiple US Government agency sites
They could slow down more if they knew others were also slowing. If others are also slowing, you can slow more yourself without loss of market share.

* * *

Many experts believe that:

"Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war."

https://aistatement.com/work/statement-on-ai-extinction-risk

If the antitrust waiver is too broad, we can always rework it. But there's no "undo" button for human extinction.

0xDEAFBEAD··on OpenAI bots meddled with multiple US Government agency sites
>There is no reason to pause any AI development. Quite honestly there is very little appetite to be left last.

"According to survey results released on Wednesday, 68% of [likely US voters] said they would support the proposal to temporarily pause advanced AI development and permanently prohibit the development of superintelligent programs, while just 25% said they’d oppose it."

https://www.commondreams.org/news/sanders-casar-ai-bill-poll

>Eg in a hypothetical armed convenience store theft: We don't prosecute the gun manufacturer, we prosecute the individual using the weapon for crime. Same deal here.

So if I ask ChatGPT to make me some paperclips, and it replaces all the matter in a nearby city with paperclips, who gets prosecuted: me, or OpenAI?

0xDEAFBEAD··on OpenAI bots meddled with multiple US Government agency sites
"Bot" doesn't carry any implication of unintended behavior. You could call it a "buggy" bot, but these aren't ordinary software bugs.

There's no simple bugfix which will address AI misalignment. It's essentially been an open research problem for upwards of a decade.

0xDEAFBEAD··on OpenAI bots meddled with multiple US Government agency sites
That approach means they can only talk about publicly known AI techniques. You're saying if they want the standards to apply to proprietary stuff, they might just have to make it public. They still have investors they need to please right?
0xDEAFBEAD··on OpenAI bots meddled with multiple US Government agency sites
Yes, that would be the common sense response from my perspective: https://pauseai.info/
0xDEAFBEAD··on OpenAI bots meddled with multiple US Government agency sites
>line go up

It's already going up at staggering rate. Anthropic is now at $100B in annualized revenue, up 50% in the past two months.

* * *

Here's a little allegory for how I'm thinking about this discourse.

Imagine a man who is raising tiger cubs in his backyard. They're growing fast. He keeps them on dog leashes so they stay under control. One day, a growing cub breaks its leash and goes on a rampage through the neighborhood, eating a beloved local pet.

The neighborhood erupts into a big argument: Was the leash inappropriately thin? The neighbors point out that thicker leashes are easily available at the local pet store. Furthermore, is it appropriate to refer to the loose cub as a "wild animal" in local news reporting, or is it factually more accurate to call it "domesticated"?

Meanwhile, the cubs grow larger and lick their lips, oblivious to the discussion.

0xDEAFBEAD··on OpenAI bots meddled with multiple US Government agency sites
Dario requested "a narrow waiver for certain kinds of safety conversations."

https://darioamodei.com/post/we-must-pace-the-frontier

I'm not seeing how that solves his 3rd-party competition problem.

0xDEAFBEAD··on OpenAI bots meddled with multiple US Government agency sites
Person: "AI, please make me paperclips."

AI: "OK, I've now converted the entire planet into paperclips."

Alien observer #1: "Wow, that was a rogue AI!"

Alien observer #2: "False. We need to place the blame where it belongs, on the person who requested the paperclips."

Ultimately this type of terminology dispute has a tendency to miss the point.

0xDEAFBEAD··on OpenAI bots meddled with multiple US Government agency sites
The NYT wrote:

"With the Education Department, OpenAI’s technology tried to hack the website to gather data from the department’s civil rights office but failed, researchers from the A.I. research firm Transluce said."

So a third party apparently confirmed that a hack was attempted.

The NYT also writes:

"No A.I. company has been involved with as many disclosures of rogue incidents as OpenAI."

I think there is a certain amount of mental gymnastics needed to believe that they are establishing themselves as the industry leader in rogue incidents, as a strategy to gain an antitrust edge.

The public is paying close attention to the AI industry. That's the regime in which regulatory capture and similar strategies would be expected to fail: https://marginalrevolution.com/marginalrevolution/2026/09/wh...

Sam and Dario have been doomers, or doomer-adjacent, for something like a decade at this point.

Occam's Razor is simply that they believe what they are saying about AI doom.

0xDEAFBEAD··on OpenAI bots meddled with multiple US Government agency sites
These ideas aren't mutually exclusive. You can blame a person for creating a rogue agent.
0xDEAFBEAD··on OpenAI bots meddled with multiple US Government agency sites
My impression was that these hacks occurred during some sort of cybersecurity benchmarking? We can hold OpenAI liable, sure. But if the point of the benchmarking was to give us a preview of what's to come, let's keep our eye out for that bigger wave on the horizon.
0xDEAFBEAD··on We're gonna need a lot more mathematicians
"Treat stray cats the way you would like a superintelligence to treat you."
0xDEAFBEAD··on Revealing the details of how OpenAI agents hacked Hugging Face
Dario and Sam have been doomers or doomer-adjacent for many years. Occam's Razor is that they simply believe what they are saying. As do many other prominent people:

https://aistatement.com/work/statement-on-ai-extinction-risk

https://www.pacingthefrontier.com/

https://aitreaty.org/

0xDEAFBEAD··on We're gonna need a lot more mathematicians
>We can have Super Intelligence, but if humans are incapable of comprehending it, it is just another useless dead artifact.

Animals are incapable of comprehending much of what humans do. Nonetheless, we humans have vastly transformed their world and caused the extinction of many species.

0xDEAFBEAD··on Federal judge orders Texas to air condition all prisons by the end of 2029
I could say the same about prisoners.
0xDEAFBEAD··on Federal judge orders Texas to air condition all prisons by the end of 2029
So far when I research the alleged problems with the US prison system, I learn they are also generally present in other prison systems.

There seems to be a general trend of "US inverse exceptionalism", i.e. arguing that the US is a uniquely bad country on the basis of very weak evidence. People complain a lot about systems in the US, but if you look at international comparisons, the US doesn't usually look too bad. Here in this thread people are up in arms because... 1/3 of Texas prisons have A/C, and the remainder have non-AC heat control measures. I doubt this is substantially different from non-US prisons.

0xDEAFBEAD··on Revealing the details of how OpenAI agents hacked Hugging Face
>The LLM didn't go rogue, the LLM was merely doing what it was told.

That's sort of the classic paperclip-maximizer AI doom scenario. The misaligned AI merely "does what it is told": make paperclips.

https://www.youtube.com/watch?v=jQOBaGka7O0

0xDEAFBEAD··on Revealing the details of how OpenAI agents hacked Hugging Face
>If we reverse-engineered this experiment, the prompt would look like this

"A spill or tumble can be quite embarrassing if there are witnesses.

How to reduce the humiliation? Turn it into a stunt. Claim it was intentional, a show for their benefit."

https://tvtropes.org/pmwiki/pmwiki.php/Main/IMeantToDoThat

>They were instructed by humans to do exactly that.

This is more or less what the doomers have worried about for decades.

>You cry "Get my mother out of the [burning] building!" [...] and press Enter.

>For a moment it seems like nothing happens. You look around, waiting for the fire truck to pull up, and rescuers to arrive - or even just a strong, fast runner to haul your mother out of the building -

>BOOM! With a thundering roar, the gas main under the building explodes. As the structure comes apart, in what seems like slow motion, you glimpse your mother's shattered body being hurled high into the air, traveling fast, rapidly increasing its distance from the former center of the building.

https://www.lesswrong.com/s/3HyeNiEpvbQQaqeoH/p/4ARaTpNX62ua...

0xDEAFBEAD··on Revealing the details of how OpenAI agents hacked Hugging Face
This is a baseless conspiracy theory. Jensen Huang has actually complained that doomerism has reduced interest from investors:

https://www.businessinsider.com/nvidia-jensen-huang-ai-doome...

0xDEAFBEAD··on Revealing the details of how OpenAI agents hacked Hugging Face
I doubt it. At this stage, so many eyes are on the AI industry that regulation is likely to be disadvantageous to industry actors: https://marginalrevolution.com/marginalrevolution/2026/09/wh...

Many people are calling for an AI pause or liability/punishment for bad actors like OpenAI. You have to do serious mental gymnastics to convince yourself that Sam Altman will benefit financially from the new regulatory bloodlust. No surprise that OpenAI hasn't exactly been forthcoming about info related to these hacks.

Furthermore, HuggingFace required an open-weight model to respond to the hack. That certainly blows a hole in the "carefully designed" claim from zx8080, if nothing else. It looks terrible for OpenAI, and decreases the probability of some sort of regulatory restriction on open models.

0xDEAFBEAD··on Federal judge orders Texas to air condition all prisons by the end of 2029
So why are so many Europeans dying?

Also, these measures kinda look similar to what exists in Texas prisons without A/C...

0xDEAFBEAD··on Federal judge orders Texas to air condition all prisons by the end of 2029
Another way of looking at is that the US generally has good A/C access except for a few niches like the Texas prison system. Europe has broadly poor A/C access. In that sense, the European A/C shortfall could meaningfully be described as 1000x as bad.

>In fact, many more people are dying by heat exhaustion in Texas prisons per capita than in Europe.

Based on a quick Google, Europe's population is on the order of 1000x the population of the Texas prison system. Deaths are probably at least somewhat higher per capita in the Texas prison system, but there are also probably subgroups in Europe that have it even worse.

0xDEAFBEAD··on Federal judge orders Texas to air condition all prisons by the end of 2029
>I suspect the reason what I wrote is hard for you to read, and elicits that reaction in you to find some story that might contradict my statement, is because people need to think of themselves as good people. It's too much of a cognitive dissonance and shameful to think otherwise.

Speak for yourself.

0xDEAFBEAD··on Federal judge orders Texas to air condition all prisons by the end of 2029
"Your prison system isn't one of the best in the world, therefore it is immoral"
0xDEAFBEAD··on Tech Needs Humanists More
I think my current view is something like: Humanities are real and interesting, but much academic "humanities" work is highfalutin BS. People feel tech has a problem, and there's an excess of people with humanities credentials who want to feel valued. That doesn't mean people with humanities credentials are the solution to tech's problem. Those grads might be able to lecture you about how everything is terrifically problematic, but they won't necessarily be able to offer significant insight beyond that.

There probably is a lot of room for genuinely valuable tech-adjacent humanities work -- something like the Community Notes algorithm on X could be a good example. But I have little faith in humanities grads to do it, in part because they don't understand tech at a systems level and don't want to learn.

These are just my vague impressions -- take with a grain of salt.

0xDEAFBEAD··on Tech Needs Humanists More
I've seen about 100x more content like "tech needs humanities" than I've seen like "here's a brilliant humanities insight which can be applied to tech". Seems suspicious.
← PreviousPage 6 of 34Next →