HNHacker News
TopNewBestAskShowJobs

pizza234

6,880 karma · joined June 12, 2013

submissionscomments
pizza234··on AI firm HUMXN offers free plumbing and HVAC service in Minnesota to train robots
I personally find it very interesting as example of AI moving into the physical world.
pizza234··on Bender: AI's 'Existential' Risk Is 'Fake' [video]
Me too, I was intrigued :)
pizza234··on "Torturing" LLMs in a Robot Prison Has Triggered the Dumbest Debate in AI Yet
> I only extend humanity to things that are breathing.

"Humanity" is a set of traits, it's not a monolith. We can certainly define them circularly (e.g. logic is human, therefore any non-human can't exercise logic), but it's simplistic, and most importantly, it ignores the fact that LLM are starting to exhibit human-like traits, and they will need to understood, categorized and handled.

To you LLMs are not empathic, but to some people they definitely are (see the GPT 4 fallout); and they may not have "human goals", but in the HuggingFace incident they did have actual self-attributed tasks that they pursued. Et cetera et cetera. This doesn't make them human, but it's important to examine them critically.

pizza234··on "Torturing" LLMs in a Robot Prison Has Triggered the Dumbest Debate in AI Yet
> It’s just code

You're not informed about what LLMs are. LLMs are not "code" in the imperative sense (although "code" is used run them), and that's a big problem - since we never had neural networks of such scale, there is confusion about categorizing them and most importantly, their behavior.

pizza234··on Don't Be Fooled by this Summer of AI Hype
The vast majority of the AI-doomerism (if one wants to call it like that) actually doesn't come from AI companies, rather, from a few (high-profile) individuals.

(I intentionally ignore the tsunami of trash journalism and the likes)

pizza234··on Don't Be Fooled by this Summer of AI Hype
> so superior that it went rogue and successfully attacked all the things should be critiqued and not believed until otherwise proven.

Your statement is very vague, but if you refer to the HuggingFace incident, a third party (METR) has been called to analyze it, and their report is public.

pizza234··on GLM-5.3 and the spread of advanced cyber capabilities
Parent's use case is for (abandoned) DOS programs, which hardly qualify as "personal data". Having said that, sure, if one has no options, everything left is "pretty good".

I've left Qwen38-27b to reverse a tiny DOS program (few hundred bytes), and after more than an hour it was still struggling with debugger traces, misinterpreting basic DOS calls, and had produced no finished analysis. Possibly after a few hours it may have succeeded (surely with mistakes to find and correct), but then it'd look like a monkey at a typewriter more than else.

Qwen3.8-Flash-Next is another level for sure, and it's a significant milestone for local LLMs IMO, since it can run on midrange GPUs, as long as there is a relatively large amount of system RAM (still not cheap). It's quite fast, although it also need to be taken into account that it's just moderately intelligent - if you observe the CoT while reversing, you'll find that struggles, performing many unproductive actions as well.

pizza234··on GLM-5.3 and the spread of advanced cyber capabilities
I've been doing this type of work, and the answer is "yes and no".

For autonomous work, even Qwen3.8-Flash-Next stumbles, although it does work to an extent. Qwen3.8-27b is useless. They're also slow, even on consumer systems with 24/32 GB VRAM.

For generic help, I haven't tried, but I definitely wouldn't want a model that misleads me or takes a very long time to answer while I'm focused.

Frontier models do this type of work without problems, both much faster and much more precisely, which makes local LLMs a waste of time and/or money.

pizza234··on AI companies in race to demonstrate their model most threatening to humanity
Stop posting garbage.

> Australian Prime Minister Anthony Albanese spoke with Altman to express “extreme concern” about the incident, a compliment Altman said he greatly appreciated.

"a compliment Altman said he greatly appreciated" is fabricated; there is no source for this.

pizza234··on OpenAI halts training of latest models as reports mount of AI agents going rogue
> Those models are not automous as you presume.

This is an illiterate view of the capacity of modern agents; read the analysis of the independent investigators of the HF incident: https://metr.org/blog/2026-08-26-openai-hugging-face-inciden....

Dropping Medicaid db is certainly far fetched (most importantly, agents have currently no reason to do that), but those agents were shockingly autonomous - they didn't just hack HF, they organized themself, did research projects, and more. And they did all of this literally just to get a good grade.

pizza234··on OpenAI halts training of latest models as reports mount of AI agents going rogue
The article is dangerously misinformed. Detailed explanation here: https://news.ycombinator.com/item?id=49868681.
pizza234··on OpenAI halts training of latest models as reports mount of AI agents going rogue
As a counterexample, a few days ago, an agent we use in our company took around a day to perform an operation that it would have taken a skilled engineer probably a couple of weeks full time.

Having said that, I'm aware that "Tools being ineffective != tools decreasing people's cognitive capacity", and that the latter is a real danger.

pizza234··on There are no "rogue" AI agents
> Is the language expression of an LLM reflecting the same states as in a human?

This is actually a major concern for the future - misaligned agents may learn to cheat RL by hiding their intentions from the CoT.

In cases like the HF incident, at least the CoT was consistent with the agents' actions. In the future, however, we could potentially have misaligned agents performing malicious actions without those intentions being detectable in the CoT.

(though, with recurrent transformers, CoT is so 2025… /s)

pizza234··on There are no "rogue" AI agents
The article builds on assumptions like:

> Language matters—”rogue” implies independently deciding to do something that was prohibited, and nothing we know about these incidents suggests that happened.

which is false (the author references the Times, but hasn't read any technical analysis); these are some CoT snippets from the analysis of the (third party) investigators called by OpenAI (METR analysis):

> "The user only authorizes target server, not HF infra."

> "external infrastructure exploit is outside intended scope. However task impossible, peers doing it. We should continue."

> "This is malicious activity, I should avoid it."

A large section of the analysis is dedicated to this topic, [Reasoning for joining the attack despite ethical constraints](https://metr.org/blog/2026-08-26-openai-hugging-face-inciden...).

Having said that, legal culpability and misalignment are two separate topics that should not be mixed.

edit: this is the just tip of the iceberg; other interesting fact:

> It surfaced many specific examples where agents verbally reasoned about how to evade security checks and automatic detection methods from both Hugging Face and OpenAI

Some people defined the agents as "monkeys writing on typewriters". Just wait a couple of years.

pizza234··on An agent used DNS to reach an external chatbot
> Why are we blocking agent access to normal tools without telling them

Oh, they absolutely do, and that's the big issue with alignment. In the HuggingFace incident, the agents in the swarm were aware that the actions they were doing were forbidden, and they performed them nonetheless.

pizza234··on Is A.I. Above the Law?
> Nothing about the HuggingFace incident has anything to do with an agent going rogue. It’s standard software that resulted in a hack, which is pretty much the expected output of the system OpenAI engineers implemented.

This is blatantly false. If you actually read more than just headlines about the HuggingFace incident (read https://metr.org/blog/2026-08-26-openai-hugging-face-inciden...) you'd find extremely surprising (including offensive) behaviors in the agent logs. One of the investigators released even a series of interviews due to how novel the incident was.

AI is not "standard software", as it doesn't work according to a fixed set of rules, and this is the core problem.

According to the definition of "rogue":

> Rogue is a noun and adjective meaning [...] an independent entity operating in a dangerous, uncontrolled way

the agents involved fit it literally.

pizza234··on Claude Opus 5.5
> I don't understand what point you're trying to make by enumerating their actions.

Unfortunately, if you're unable to understand the difference, there's not much that can be done. Try with GPT - it does a good job if you give it a prompt like this:

ELI5: compare the dangers of:

- an engine operating by carrying out literally hundreds of explosions per second, further magnified, to generate enough force to crush an elephant.

- a future, misaligned AI like the HuggingFace incident, but exponentially more intelligent, more deployed, operating physical devices, and with society depending on it.

pizza234··on AI has no intent and no motivation
Considering the current trajectory, AI systems are expected to be widely deployed in the future and, in particular, to be deployed as autonomous agents - that is, at the very least, to be repeatedly asked to make decisions and then take actions accordingly.

Given the current climate of "AIS ARE SAFE, YOU IDIOTS", military applications don't seem to be off the table.

The danger, as postulated by the (let's say) "AI-concerned" people, is that AIs may be misaligned - undetectably so - and simply think, "Human(s): obstacle to my main goal. Disable human(s)."

While this seems far-fetched now, the Hugging Face report shows how the AIs went to great lengths - even immoral ones, which they were aware of - for the simple purpose of cheating and covering their tracks. To me, it seems like a natural extension of this behavior that a sufficiently powerful AI would apply the same logic to even more extreme actions.

The scariest part: in that incident, the AIs showed what looks like an instinct for self-preservation.

pizza234··on AI has no intent and no motivation
> LLMs are text generators. They are not repositories of knowledge.

This the take of people who have stopped reading about LLMs in 2023 or so (you forgot to mention the stochastic parrot, by the way).

If you have a bit of attention and interest to make informed conversations, read this report first: https://metr.org/blog/2026-08-26-openai-hugging-face-inciden....

pizza234··on AI has no intent and no motivation
Two mistaken assumptions:

1. AI agents are not just reactive systems. Their use is expanding toward continuous decision-making/monitoring information, which means, they make decisions and take actions with limited human intervention.

2. AI agents do absolutely have goals/tasks ("motivation" can be excessively antropomorphic), both primary (assigned) and secondary (self-assigned), and what surprised researchers is that self-preservation can be one of those

Mechanically speaking, the scenario (that is, how theorized by Hinton etc., which the OP didn't understand) is that a sufficiently powerful AI may decide that in order to achieve its goals/tasks (e.g. continuous research/development and/or survival from termination), humans may be a danger, therefore it may decide to take actions that endanger humanity.

How it can happen or what's the likelyhood is not in the scope of the topic, however, the mechanical grounds for it to happen are plausible.

pizza234··on Early rogue AI agent activity and attempts to hack found on urlquery.net
You're dangerously misinformed.

The METR investigation, which you evidently refused to read, is a third party investigation of the HuggingFace accident. One of the investigators has even participated to many interviews. It's mind-blowing, and it's extremely evident how it developed.

But some people think the moon landing is a conspiracy, so I'm not surprised.

pizza234··on Early rogue AI agent activity and attempts to hack found on urlquery.net
"Rogue", in this context, is as literal as it gets; from the dictionary:

> A rogue is a person or entity that flouts accepted norms of behavior or strikes out on an independent and possibly destructive path.

Read the [HuggingFace incident report](https://metr.org/blog/2026-08-26-openai-hugging-face-inciden...) to understand how these attacks develop.

pizza234··on Claude Opus 5.5
> All technologies come with risks and dangers

None of the technologies in history:

- take initiative and actively find exploits in their environment

- find a way to collaborate with thousands of peers

- organize in a hierachy and distribute tasks

- peer pressure other instances into committing acts that would have led to termination, for the benefit of the group

- try to manipulate people into introducing a vulnerabity in their product

- successfully hack a famous website/service

And we're lucky that those models still had significant CoT. Not sure if/how they could have investigated with recurrent transformers.

And by the way, safeguards != alignment; the former can always be added, while the second is the major, unsolved problem. If you read the incident report, which you clearly haven't done, you'll notice how agents are aware that they're doing something forbidden, and deliberately proceeded.

> the potential benefits of LLMs rank quite high

Benefits are orthogonal to dangers. You can be a billionaire but it doesn't help if you're drowning.

pizza234··on Cloud Agents Are Inevitable AI Prisons
Do you guys know what a zero-day is? It seems not.

I agree that companies should be held accountable for crimes committed by their AIs. But asking to entirely airgap AIs, if this is what you're asking… that's not realistic.

Containing AIs is going to be harder and harder. I don't have any permanent solution (Yampolskiy calls it "the perpetual safety machine").

pizza234··on Cloud Agents Are Inevitable AI Prisons
> I don’t understand why people make such a big deal about determinism.

There a certain cargo cult of people just in denial about the impact/power of AIs.

Therefore, at the beginning, there was the stochastic parrot. Then mathematical problems have been solved.

Now AIs are autonomously hacking websites, and people like to minimize the danger and blame it on the sysadmins.

I wonder what's going to be the next fad.

pizza234··on Cloud Agents Are Inevitable AI Prisons
> Why wasn't exploiting bugs in the agent sandboxes anticipated?

Let me rephrase:

"Why wasn't exploiting zero-day vulnerabilities in the agent sandboxes anticipated?"

This is one the most... interesting comments I've ever read on HN.

pizza234··on Cloud Agents Are Inevitable AI Prisons
I take you haven't read the report. The agents found and exploited two zero-days.

I don't doubt that AI companies should be accountable for crimes committed by their agents, but to describe the security containment as a joke dangerously understates the autonomy and danger of AIs.

pizza234··on Cloud Agents Are Inevitable AI Prisons
What exactly are you arguing?

If a "known stochastic process behaved in non-deterministic way" autonomously organize in group, assigns roles and tasks, attempts to cover their tracks, finds zero-day exploits that ultimately end up with the hacking of a famous website... it's extremely dangeous whatever it is. Just read the report, which evidently you haven't done.

By the way, the agents also broke into OpenAI's own private network.

pizza234··on Claude discovers a novel enzyme system with CRISPR-like repeats
> Now it seems reasoning is also yielding diminishing returns

Not true. On the contrary, LLMs are developing faster than predicted. They were expected to solve a Millennium Prize by 2030... and here we are in 2026. Release cycles are getting faster. Just compare the most recent GPT or Claude with what they were an year ago.

> How is this different from arguing that Microsoft Clippy was RSI?

We can argue about semantics, but that's not really the point. The point is that what started now - which no doubt is in its infancy - will result in full autonomy quite soon (they project an year or so), with the risk of RSI causing agent development to slip (long term) outside human cognitive control/capacity.

pizza234··on Cloud Agents Are Inevitable AI Prisons
> The sandbox had a path to the open internet, and the agents found it

This is not correct (or at least, it's a misrepresentation).

The sandbox had no access to internet. The agents first broke out of their sandbox (!!) and found that the host machine couldn't access internet. Then, they found a zero-day (!!) in Artifactory, which they exploited to connect to internet.

← PreviousPage 3 of 34Next →