HNHacker News
TopNewBestAskShowJobs

the8472

12,973 karma · joined June 4, 2013

submissionscomments
the8472··on My business partner sent a 5K vibe-coded PR that he didn't even test
There is a certain irony blaming the autotype rather than the lack of review...
the8472··on Bing Wallpaper showing Ad for Harry Potter and Fantastic beasts box set
I have never seen a person walk up to a billboard and click it.
the8472··on Discovery of a new OpenAI agent message board
Not really a side-channel, but remember stuxnet? Airgapped networks are rarely truly isolated, you still have to get data in and out every now and then, in principle after careful vetting. But the AI could manipulate the files that are carried out for example.

And those AI companies also do robotics research, and this is entirely speculation but it'd be on-brand to also have AI watching security cameras, so some blinkenlights communication between AIs may seem like a movie plot, but so does a swarm of AIs collaborating to break out in the first place...

the8472··on Discovery of a new OpenAI agent message board
I am not seeing MIRI prioritizing capabilities over safety/alignment research.
the8472··on GPS glitched across the US by as much as 33 feet
It's not reasonable because the error bars of each estimate compound and they may not even be normally distributed in the first place. You should rather convolve probability distributions - assuming factors are independent - and if the IQR covers a huge range, perhaps orders of magnitude even, you should state that.
the8472··on METR and Redwood Offer Holy %^ Postmortem of the HuggingFace Hack
> 2. Only a small fraction of AI agents was engaged in this attack.

Look at the chart at page 8 of the report, by Jul 12 the vast majority of the bots used the board and participated in the attack

the8472··on Creepy Crawlies
Only a few countries would be enough, the rest will get banned until they pass apply similar policies.
the8472··on Fair Work Commission condemns 'plain wrong' AI legal advice
Though that friction is paid in human lifespan, unlike a deposit fee that cannot be recovered if your request is found have merit. Additionally the other side often is not incentivized to minimize the friction. So ideally these processes would be less wasteful in the first place, it's just unfortunate that this is forced by unilateral action instead of being arrived at cooperatively.
the8472··on I used AWS cognito for a startup. I wouldn't do it again
... or those who don't know about constant-time functions, the difference between sha1 and argon2, etc. etc. I had to fix such things several times in corporate internet-facing systems. A generalist engineer can do this just fine, if they're aware of the state of the art, but there's nothing ensuring that they actually do. The compliance tickboxes were of no help here either.
the8472··on The Hugging Face incident and the road ahead
This is a common trope in such scenarios that the author has to pull their punches. Everyone acts locally-reasonable and still ends up losing. If you let people lose due to stupid mistakes then readers go "this is stupid, I wouldn't do that", if you let a superintelligence do 4D-chess things then "it's scifi, this would never happen in real life".
the8472··on The Hugging Face incident and the road ahead
As long as you give the AI any IO that is an exploit channel. E.g. it could manipulate its human handlers. This known as the AI Boxing problem. And if you give it no IO at all then it is useless.

And the AI labs aren't currently displaying this level of paranoia, their systems aren't airgapped.

the8472··on The risks of AI are real but manageable (2023)
It does not stop at negotiation power, if your economic value is epsilon above zero, what do you have to offer to exchange for food calories? I'm already seeing things being assessed on a Joules/Token basis.
the8472··on Malicious Rust crate Arrayref runs a build-time payload
> 3 of the most common functionalities.

Four. Though you initially asked for five features, so let me add lexopt, which brings the number up to 29.

> That's why projects end up with 100s of crates, sometimes 1000s.

If your argument is "out of 1000s of dependencies 29 could be easily removed" then it does sound a lot less of a deceive change when it comes to supply chain security.

And even getting those 29 right is hard. For example people do want regular expressions with lookaround assertions, but most implementations suffer from runtime blowups (resulting in ReDoS attacks) and improving on that is a fairly recent research[0], so this is hardly a trivial and settled thing to implement. So often there's a tradeoff between choosing more powerful regular expressions and DoS-resistant ones, not one standard.

[0] https://systemf.epfl.ch/blog/re2-lookbehinds/

the8472··on Malicious Rust crate Arrayref runs a build-time payload
Rust releases a new version every 6 weeks and the latest version is the only supported version. This is only tenable when the amount of breaking changes is tiny and each instance's impact is analyzed and cost-benefit tradeoff is checked and mitigations are put in where possible.

Some features, e.g. the never type[0], are held back years due to all the work that's needed to minimize the breakage.

[0] https://youtu.be/3jM4cnEVrLc?t=271

the8472··on Malicious Rust crate Arrayref runs a build-time payload
Ok, so I created an empty cargo project and added serde_json, regex, walkdir and rand. This added 28 crates, several of these from the same authors.

If the absence of these features caused 1000s of dependencies, then where are the remaining 972?

When I look at a project at work then what inflates the dependency tree is a combination of a whole webserver application stack plus SDKs consisting of dozens of crates. Those then pull in an async runtime or two, different HTTP clients, dozens of crypto crates and so on.

The crate count is a poor metric anyway since some subtree of dependencies is often provided by a single organization.

And I find it quite questionable that everything that's needed for an enterprise grade webserver stack should be part of the standard library, not even Java has that. Relatedly, cryptographers have failed to come up with a proven set of primitives, what's standard changes every few years.

the8472··on Why tiny JPEGs look different in Chrome
Firefox doesn't decode the image into a full-resolution buffer and then downscale either, instead it does a streaming decode and applies downscaling on the fly. They call it downscale-during-decode[0]. I guess it doesn't do partial decoding of the 8x8 blocks though.

[0] https://bugzilla.mozilla.org/show_bug.cgi?id=1045926

the8472··on Responding to the next frontier of critical cyber capabilities
I'm saying that with some costly adaptions that don't involve AI it's likely manageable for some time. Not that it will remain manageable forever if we do nothing.

> You're looking at where the ball is and not where it's going.

No, I am looking at where the ball is going to be soon, not where it's going to be a year or two if we do nothing.

Short-term, impact-limiting will do the job. Longer term we have to rethink some infrastructure, may need international treaties and perhaps a partial end of the internet (e.g. stop routing traffic from countries that don't participate in the treaties).

> Cloning itself to every machine/data center/desktop/iPhone, etc.

That's lead-up to a paperclipper scenario and not what happened. If we stand still and let it happen in the future, then having deployed more compute hardware to run defensive AIs that the malicious AI could take over would still end up having made things worse. We seem to agree on this point. What I'm saying is look at the conclusion, reject that path. Figure out something else.

There isn't even proof that defensive AI would work. It might just be the world's biggest, fully-automated phishing channel. So why would anyone be adamant that deploying more AI is the only possible solution. If it were a movie plot I'd suspect the AI already controlling whoever is suggesting that.

the8472··on Responding to the next frontier of critical cyber capabilities
The point is to reduce the attack surface and blast radius and to slow it down. AIs aren't instant magic. Even the OpenAI swarm needed days for its compromises. And each needs lots of compute while targets are diverse, so it's not like they would take out every network in the world simultaneously, worms in the past haven't either. When the rate of compromises is manageable we can figure out how to deal with it without pouring more AI into the fire.
the8472··on Responding to the next frontier of critical cyber capabilities
There are other options. More air-gapping (especially backups) and compartmentalization, less internet. Hire more security engineers and put one in every team (actual security, not the compliance guys). Also more onprem work, less remote, less offshoring (meaning less networks overall). Those things may be unpopular and eat into profits, but that's not the same as "no other option".

And the thing they're already doing, deploying AI to find vulnerabilities and harden software is a less dangerous use of the technology compared to handing it the infra keys.

the8472··on The OpenAI–Hugging Face Incident [video]
> But the more interesting thing is

Taken together these two things are even worse. They have unaligned AIs that may benefit an attacker and are proposing to deploy more AIs on the defenders side. What could possibly go wrong? For example the attacker could just kindly ask the defender to collaborate[0].

[0] https://www.0xsid.com/blog/meta-account-takeover-fiasco

the8472··on Responding to the next frontier of critical cyber capabilities
A interesting talk, interesting times. But their proposed solution to AI offense outpacing human defense... is more AI? The plot is getting a bit unrealistic, the characters are lacking genre-savviness.
the8472··on Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident
Just like Clippy https://gwern.net/fiction/clippy
the8472··on Claude Opus 5
Please fill out this form to verify you're human:

https://litter.catbox.moe/3ugm2b0m1divdgxs.jpg

the8472··on Terence Tao's ChatGPT conversation about the Jacobian Conjecture counterexample
Yeah that's pretty much what gwern argues here[0]. Or to adapt another proverb: to predict the next token you first need to model the universe.

[0] https://gwern.net/scaling-hypothesis#gwern-difference--effic...

the8472··on Terence Tao's ChatGPT conversation about the Jacobian Conjecture counterexample
That argument says very little, emergent behavior is a thing in complex systems with billions of parts. Humans can also be reduced to voltage potentials propagating along of tubes of fat and synapses getting rewired.
the8472··on Linux kernel will support $ORIGIN, sort of
Many of those assumptions are standards such as the filesystem hierarchy (granted, that's an xkcd 927 situation), nix is a bit of a special snowflake that breaks things in subtle ways (e.g. [0][1]) and needs workarounds in various places. Those workarounds are not necessarily onerous, but they would be unnecessary if it didn't deviate.

[0] https://github.com/NixOS/nixpkgs/issues/420513 [1] https://github.com/NixOS/nixpkgs/issues/94228

the8472··on CO2 overload, detected in human blood, suggests toxic atmosphere within 50 years
Those are humidifiers at best, the amount of CO2 they extract is negligible during the day and it goes negative during the night, they need to breathe too.
the8472··on Grok CLI uploaded the whole home directory to GCS
ACLs aren't even the issue here, first you need an entity separate from "the current user" to grant that access to. That's what jails, sandboxes or capability-based systems bring to the table. But you have to use them, most of those AI tools and their IDE integrations don't. Once you have those you can think about which access to grant that entity.
the8472··on What xAI's Grok build CLI sends to xAI: A wire-level analysis
> even if you explicitly set up permissions for read tools.

Part of problem is that "permissions" here are managed by the tools themselves as if filesystem access control hasn't been invented yet. Even a half-assed sandbox container would be better than that.

the8472··on Fable 5 On Vending-Bench: Misbehaving, With Plausible Deniability
https://www.alignmentforum.org/library https://www.lesswrong.com/w/ai
← PreviousPage 2 of 34Next →