HNHacker News
TopNewBestAskShowJobs

kalkin

1,258 karma · joined November 2, 2012

submissionscomments
kalkin··on We must pace the frontier
Sure. And there's an OOM more compute coming online in the next year or two, while models at a given capability are getting cheaper. "Two OOMs" of scale relative to the HF swarm seems like a bit of a red herring to me, but also within the realm of possibility.

The Internet is big, but one can do quite a lot of damage with ordinary bots and worms that exploit individual widespread vulnerabilities, which LLMs are perfectly capable of writing. Most of the damage also doesn't rely on hitting every long-tail website.

I'm honestly not that concerned about cyber impacts of LLMs relative to other impacts. I just don't like to see the whole concept of being worried dismissed as obviously baseless on the basis of one pretty shaky scale argument.

kalkin··on We Must Pace the Frontier
I'm curious to hear more about the differences in how bioweapons are regulated vs nuclear weapons.
kalkin··on We must pace the frontier
If Anthropic shut down (or even counterfactually had never been founded), would software development be freed from the impact of AI?
kalkin··on We must pace the frontier
Export controls vs China don't imply Dario's personal control of AI, nor are they broadly in the interest of "tech billionaires" (Jensen hates them). Neither does admitting external auditors at AI companies--if anything that's a surrender of control.

I don't see anything in the essay about restricting open frontier weights.

If you want to argue that the US has no right to be restricting technological development elsewhere, you have a case. But why tie this to free-market pseudo-populism against basic, domestically-scoped safety regulation?

kalkin··on We must pace the frontier
Why should we believe that a scaled out version of something that happened a few months ago "simply cannot happen"? How many dollars of compute do you believe were available to the swarm(s) behind the OAI-HF, German wiki, and Rubygems incidents?
kalkin··on We must pace the frontier
Where in the essay does it propose "The state must use its coercive power to support my control of this technology to the exclusion of [people not like me]"

I don't think that people outside of tech think the problem with "tech billionaires" is that they occasionally support some limited regulation. There's a weird form of tech populism that takes as axiomatic that any government intervention is "regulatory capture" and insists the only way to combat the power of big tech is unfettered capitalism that seems bizarrely prevalent on HN given how little sense it makes in a normal political context. Like, Bernie Sanders' position here is simple to understand: ban it. But on HN you'll see people framing the side with Marc Andreesen and Peter Thiel on it as against "tech billionaires".

kalkin··on Claude is only available to people over 18 years
If this is posted because it's "still relevant", the headline probably shouldn't say "no longer".
kalkin··on Claude is only available to people over 18 years
That looks like when the support page was last updated -- was Claude even available for minors before that?
kalkin··on Discovery of a new OpenAI agent message board
> Why and how it happened is irrelevant

Do you take this attitude for other things that negatively impact society, or is it reserved for cases where it's particularly important for our comfort to deny that anything novel or scary could be involved?

kalkin··on Discovery of a new OpenAI agent message board
They literally hid this until a third party reported it.

This is chemtrails-level conspiracy theorizing at this point.

kalkin··on METR and Redwood Offer Holy %^ Postmortem of the HuggingFace Hack
In that hypothetical 3-years world, should we be _less_ worried about aggressive behavior by agentic AI systems acting against the intentions of their developers? I don't follow how your scenario is supposed to be an argument against worrying about alignment.

e: I do actually get how worrying about emissions or child safety or concentration of wealth might be competitive with worrying about alignment. I don't see how you have the worry "AI is very close to being able to power autonomous drones that could kill us all" and then see control of those drones as a non-problem.

kalkin··on Creepy Crawlies
> zero intelligence here

That's head-in-the-sand stuff. AI is certainly very capable of being dumb (as are humans). But:

> “The problem was in need of a new real idea, which this new result seems to provide,” says James Maynard, a mathematician at the University of Oxford. “It seems that the AI has made a genuinely interesting mathematical contribution.”

https://www.scientificamerican.com/article/no-ai-didnt-just-...

Nobody a decade ago would have said "oh yeah solving a bunch of open problems in research mathematics, and finding a bunch of zero days in Chrome and Firefox, and winning literature prizes, are things that don't require intelligence."

kalkin··on METR and Redwood Offer Holy %^ Postmortem of the HuggingFace Hack
Recall that the agents in some cases found sandbox escapes. Although, with the specific example of spoofed tools, it's unclear if that was necessary--it appears that they were able to create tools (CLI tools within the sandbox?) that took precedence over normal tools and did something different while looking identical in (a local portion of) a transcript. I'm not sure I'm getting this correctly but it seems like this might have only required the ability to add things to their PATH which they plausibly have inside a sandbox, and then the transcript doesn't need to be tampered with directly.
kalkin··on Creepy Crawlies
This tick about "if AI smart how come crawler dumb" is in most complaints I've read about AI crawlers and I've started to find it pretty annoying. The crawlers might be written using AI but they're evidently not actually running AI inference over the pages they get back--besides being able to tell this from the behavior, if this is pretraining input, that's enormous scale, so it'd mean a large increase in effective training cost. Naively assume inference costs are equal to pretraining costs (probably not true but maybe right order-of-magnitude) and it's a doubling.

This ends up tacitly turning a very legitimate complaint (ill-behaved crawlers) into a justification for head-in-the-sand AI denialism.

kalkin··on The Hugging Face incident and the road ahead
> I do believe that big tech companies do not release all the postmortems publicly

Right, but when they do release postmortems, do you think it's "marketing"? Where they're actually exaggerating how bad the incident was because there's "no such thing as bad publicity"?

kalkin··on The Hugging Face incident and the road ahead
I've never heard of Intertel, but Wikipedia says:

> Nixon's staff also anticipated that the Democratic campaign would employ the services of Intertel

Are you sure you're not garbling the story?

In any case, I would expect an ethical firm to refuse to spy on the president's political opponents and want one that broke the law to be prosecuted, but more importantly, the gaping hole in your analogy is that Nixon directed spying _on his opponents_, but OpenAI did not direct hacking _of HuggingFace_.

What you're doing is more like saying "the American people elected Nixon with a mandate to spy on enemies, so what right do they have to complain?"

kalkin··on The Hugging Face incident and the road ahead
> I wouldn’t hire them again

Right, me neither. Because there's a common sense delineation between actions that are reasonably expected when "a system was given a goal and it achieved that goal" and actions that are obviously misaligned with the goal-giver and unwanted even if some indirect sense they were causally related to the goal. We have no trouble making this kind of distinction for humans, so we shouldn't pretend it's impossible for AIs in order to put our hands over our eyes and pretend there's in principle no such thing as one that's misaligned or rogue.

kalkin··on The Hugging Face incident and the road ahead
> they have chosen to because it benefits them

Or perhaps they've chosen to do this because they feel they have a responsibility to do so.

We understand this when tech companies publish postmortems of outages and security incidents--that it's an attempt to fulfill an obligation to users and the industry (and in some cases regulators), not marketing about how in-demand their product is or something. As far as I can tell we generally accept this as a default hypothesis even from companies led by people like Elon, Zuck and Kalanick--in part because we understand that these companies have thousands of employees, most of whom aren't marketers. Why are we uniquely conspiratorial about OpenAI?

kalkin··on The Hugging Face incident and the road ahead
> a system was given a goal and it achieved that goal

If a security firm you'd hired for pentesting did this (hacking a third party, and not informing you and covering it up), would you hire them again? Or would you say it was your own fault for giving them too broad a goal?

kalkin··on Guess which of these LLM outputs is watermarked
How would you re-introduce correlations on top of a CSPRNG without a cryptographic break?
kalkin··on Guess which of these LLM outputs is watermarked
It sounds like you have a publishable paper - you've broken Aaronsen's watermarking scheme. Congratulations!

(Or maybe you got lucky.)

kalkin··on OTel isn’t going well
The article assumes the issue with OTel is slow feature development, which isn't my experience at all. The issue I've had is that the SDKs have terrible performance overhead for instrumentation and are, as you say, highly resistant to integrating the output of better performing (or just preexisting) instrumentation. In Python and Ruby, at least, the CPU cost of all the mandatory abstraction is way too high.
kalkin··on Pacing model development in an era of cyber-critical capabilities
All we need to retain human control over AIs is for nobody to write any bugs. Piece of cake.
kalkin··on Pacing model development in an era of cyber-critical capabilities
If logs are eventually released that are basically consistent with OpenAI's story, are you planning to adjust your approach for judging what's only a "sci fi plot" and what could actually happen? Or will extrapolating anything beyond what's already been definitively proven be "sci-fi" still?

Not that you should need logs. OpenAI is a company with thousands of employees, very few of whom have "billions in options". If they were just making it all up, it would leak. (OpenAI is notoriously leaky!) Not to mention, HuggingFace would not have reported it to the police (apparently before they knew it was a rogue model). jFrog would probably not be playing along quietly with a claim that Artifactory is full of zero days. The UK's AI Security Institute would most likely not have published a report about analogous behavior by Anthropic models. The idea that talking about your product's dangers is good marketing never really made any sense, but even if you were going to do so, why would you include as many frankly embarrassing details as OpenAI has disclosed?

The evidence is only weak by absurdly selective standards that would have you doubting basically everything you might read in the newspaper. A healthy skepticism is one thing, and head-in-the-sand denial is another.

kalkin··on Pacing model development in an era of cyber-critical capabilities
> fable is a world ending event

Did anyone actually say this?

Mostly what I have seen is people saying "hey at some point these models might get dangerous." And the type of HN commenter who mistakes blind cynicism for wisdom laughs that off as marketing. And now when (some) worries appear to come true, somehow having previously expressed those worries is not being proved right, but in fact discrediting, because it was "crying wolf."

> simultaneously spinning down expenses

Unless OpenAI is renting their compute to others, spinning down RL training doesn't save them any money.

kalkin··on Security Incident INC-2026-07-28-01 – UK AI Security Institute [pdf]
I wonder if HN is also going to insist this is just marketing for OpenAI and Anthropic, or at least good PR for them somehow.
kalkin··on Our position on open-weights models
Wouldn't your proposal also amount to a ban on open weights models? At least for any developer that isn't unshakably confident that no court will ever find their model to have done significant harm?
kalkin··on Be skeptical of OpenAI's rogue hacker agent story
> In 2016, the Chicago Cubs won the World Series

> This broke a hundred-year drought

> A few days later, Donald Trump won the 2016 presidential election

It's not a 100% gold standard RCT or whatever...

kalkin··on Be skeptical of OpenAI's rogue hacker agent story
Let's start by noting you made a factual claim that you apparently had no backing for whatsoever. I'd recommend against doing that.

> they would be perfectly capable of keeping it just between themselves and not putting out press releases about it.

In some jurisdictions there are legal requirements about disclosing cybersecurity incidents, and it's a well-established best practice even where there aren't. Beyond that, HuggingFace appears to have put out a press release about this before they knew this was a rogue OpenAI model: https://huggingface.co/blog/security-incident-july-2026

Of course, if it's all a big conspiracy, perhaps they were just not saying what they knew in order to help OpenAI's marketing later. Or it never happened at all, or whatever.

I do disagree that this is good PR, but I don't see the point in having that argument here--I just want to correct straightforward misinformation.

kalkin··on Be skeptical of OpenAI's rogue hacker agent story
> this outcome is good for their business and stock price

You state this as fact--how do you know it? OpenAI isn't publicly traded. On Hiive OpenAI is marginally down in July: https://www.hiive.com/securities/openai-stock

← PreviousPage 2 of 13Next →