Comment by Scryptonite | Hacker News Reader

Comment by Scryptonite | Hacker News Reader

Parent Full thread

I don't want to come across as inflammatory, but I'm not sure I really align with the folks here who want retribution and the law to come down hard on OpenAI (and related orgs). We are surrounded by vulnerable systems, and AI has alignment issues that need to be investigated and sorted, and it just seems like the hullabaloo is from fairly innocuous breaches. Does it make sense to exert so much effort in preventing these events/learnings by well-intentioned organizations? I think we should be more grace giving for these events. As another poster pointed out, CFAA specifically addresses intentional hacks, and I think it is impossible to meet that criteria with these events. I also really don't understand the 'well, the law came down hard on small individual hackers [who meant no malice], thus we should apply it to large orgs [who also mean no malice]".

It seems like playing it too safe leaves systems vulnerable, and threat actors will inevitably possess ever more similar abilities, and the breaches will happen one way or another, possibly with less or zero accountability if the threat actors pull it off with greater precision. If I were in charge of the law, I probably would begin to incentivize for trusted actors scaling up and aggressively searching for these types of weakness, and privately disclosing them ASAP, so that systems can be patched.

What if a company had human developers doing these same things? What would happen to those developers, those companies? Would anyone be so blasé? Just because it’s AI, it should be forgiven? How is that just?

And don’t forget, OpenAI is supposed to be a non-profit / public benefit corporation creating AI for the good of humanity. These incidents are clear evidence that they are being extremely reckless and are not living up to their supposed standards. What does it matter if they “mean no malice”? They are in a blind rush for profits and that has made them reckless and dangerous. They cannot be trusted.

The company should be shut down or fined into oblivion, as a warning to other companies to not pursue the same reckless path. If they get away with this, then that risks normalizing this behavior.

> by well-intentioned organizations

Your entire comment hinges on this assumption and I don’t think it’s sound. Their intention is to make money at any cost.

That seems potentially reasonable for organizations who have have a unlimited profit motive, but I'm not sure how you would insist that is the case for OpenAI. They are governed by their nonprofit body, their mission is to ensure AGI benefits all of humanity, and they've said (Sam Altman) that if there is a choice between safety and profits, profits take a back seat and wait. Is the trick that we just insist that they are actually only caring about the money, and they actually have no genuine concern/effort regarding safety?

And if none of the frontier labs are as you say, it doesn't seem like we're negotiating for the best future by restricting labs from stumbling into these vulnerabilities full-stop. The vulnerabilities don't go away unless we find and fix them.

If the goal is to ban open models, then it doesn't matter what the motivations are. The people advocating for restrictive releases haven't substantiated their stance at all. We've had Mythos-tier cyber capabilities available for pennies on the dollar, for months, and there's no red mercury bomb or bioweapon to speak of.

We've seen OpenAI's nonprofit board do ridiculous things in the past. I can see how they would be desperate to retain the disproportionate amount of de-facto mindshare and authority they enjoyed a few years ago. Especially if the future entails more ugly US government contracts that undermine their ethical sales pitch.

That's okay. It isn't necessarily incompatible with being well intentioned regarding the disclosure of vulnerabilities.

On the whole I think you could describe the local big box grocery store as just wanting to make money yet as a direct result of that they are well intentioned from my perspective with regard to the food they sell being edible.

You’re right that punishing unintentional or well-intentional hackers will actually empower ill-intentional hackers.

However 99% of businesses will not be able to effectively handle security reports in reasonable time. They did not need that level of competence or funding until now, and therefore they did not acquire it. Most companies that use locks don’t employ locksmiths.

Looking another way - the entire economy got suddenly hit with entire pile of technical/ security debt it has accumulated over 50 years, and only has one year to deal with it. No amount of diagnostic will fix this, this is economically infeasible.

One thing that is feasible is a government bailout for technical debt. All the engineers that were freed up from AI-induced restructuring should be employed to pay down technical debt across the entire economy. Simply updating all software to use latest libraries is a great start. Using AI to diagnose, repair, re-deploy all old software. Using AI to rewrite all old software that cannot be recompiled.

I'm old enough to remember when innocuously poking around other systems was a crime, and many a curious person were charged and found guilty. But if I write a program to do it..?

I am familiar with this also, but it doesn't seem like the implementers of that apparatus/value-system made the absolute best choice balance in terms of long term safety and security our computer systems. Is the conclusion about this rhetoric that it is right to punish innocuous poking of systems? Or the inverse? Which would you have preferred if we had an opportunity to refine it?

   Does it make sense to exert so much effort in preventing these events/learnings by well-intentioned organizations?

No, but what I think you’re seeing is many people who do not believe OpenAI is a well-intentioned organization. For example, their view might be that if we did as you suggest and incentivized OpenAI to search for these weaknesses, they would, but then they would not “privately disclose them ASAP”. (And I think they’d have a point worth engaging with, when you consider how they slow-rolled the HuggingFace stuff so much that independent researchers were still discovering new instances of the attacks months later).

What else can you do with (what you believe is) an inveterate bad actor?

Half of HN hates the big labs. All their “reasoning” stems from there. Don’t try to make too much sense of it.

No.

If this is endorsed and thus malicious, they should be ground into dust by the law.

If this is accidental and due to incompetence, they should be ground into dust by the law.

I've secured many, many far less important and impactful systems substantially better than even the best sandbox environment that we're aware of being used by these labs.

There's simply no reason to let these things interact with the internet at large. We can already rapidly copy existing tools and services (albeit questionably) using these models, why not simply run in a fully virtualized, air gapped environment?

Because they don't care.

This is incredibly blatant intentionally malignant behavior. The fact that so many people question this is frankly stunning.

They're getting away with it simply because they can point at the other guy and say "see? they're doing it too!"

It's ridiculous and they should all be litigated into a fine residue.

We take biological lab leaks seriously, how is this substantially different?

Well, the COVID story seems to indicate we don't actually take lab leaks seriously, so... yeah.

Reply on news.ycombinator.com