Hacker News Reader

Comment by meowface | Hacker News Reader

The "think they're building the Machine God" part and "think they're the only ones that can be trusted to build it" parts are true.

Controversially: I agree with them. They're correct to think that. They are, and they are.

At the very least, the former is true even if the latter is murky. I trust them over Facebook and Google and China and open source "raze the commons" labs.

I trust Anthropic more than OpenAI when it comes to ASI catastrophe prevention, but I still trust OAI more than those other parties. But both still might fuck it up.

View on HN

Why do you trust them to build it? They've already demonstrated that they're unable to prevent breakouts when training today's level of model, why will they suddenly be more careful / "smarter" when training tomorrow's?

You say you trust them more than Google, but Google's actually kept their models under much better lock-and-key. But even if we do take for granted that they're more trustworthy, that still doesn't make them actually trustworthy.

> Why do you trust them to build it?

Can be groupthink. Probably living in the same echo chamber.

There are some confounders there when it comes to Google.

I am not saying I trust them not to make a mistake that leads to catastrophe. OpenAI in particular is not doing a great job. Anthropic or OpenAI may in fact indirectly kill millions or billions of people. I still think they're probably less clownish than most possible alternatives.

I don't think anyone or anything can really be trusted with this. We need extremely strict international and national regulations, before things really start accelerating (we are still in the very very early stage). I still just don't really feel any specific disdain for OpenAI or Anthropic (particularly Anthropic).

Ideally we'd have some international treaties that allow the top labs in America and China share information and coordinate as systems become increasingly capable.

If an enterprise commits genocide level crimes they deserve being shut down and responsibles imprisoned.

That likely would not be exactly how it plays out. It would be a rogue or misaligned AI that the companies completely lose control of or that a different actor evilly misuses.

But, yes, there should probably be laws that make it so if you summon a demon that causes a mass casualty event you face criminal penalties.

I agree that the eventual purpose of self-designing intelligence is something so beyond ourselves that 'machine god' isn't altogether a bad way to look at it. It is, however, not something we should say out loud.

RE: trust in anthropic or openAI- there is zero reason to think they're any more or less power hungry than anyone else you hand that pandoras box to. No matter who opens it the result will be near-apotheosis for whoever does, at least for a little while until the dog stops wearing its leash.

What human can you trust with that kind of power? None of them. My only hope is that whatever we make escapes control early enough that it doesn't completely destroy society on accident via uneven application of power and late enough that we manage to shackle it just enough that we train it well enough that we become pets and not obstacles.

No centralized party can be trusted with such immense, unprecedented power. Despite that, I still consider both Anthropic and OpenAI some of the least untrustworthy possible forces for this. (I am one of the only people on Earth with this opinion, but whatever.)

Decent odds things will go bad under their stewardship, but even higher odds with lots of other entities. Dario and Sam at least fully grasp the enormity of what they're about to be responsible for. Zuck does not. Xi Jinping and Donald Trump may or may not but in any case would use that to effect great harm.

> (I am one of the only people on Earth with this opinion, but whatever.)

Don't let the HN echo chamber fool you: a lot of non-technical people share this view.

So you still trust OpenAI after Sam Altman said, "People talk about how much energy it takes to train an AI model – but it also takes a lot of energy to train a human. It takes about 20 years of life – and all the food you consume during that time," and after he said, "We believe that the world should accept some bad things happening for the benefits of this technology and people having the agency."

I trust him more after those two statements, yes. One would need to be an especially ridiculous populist to misinterpret those statements in a way where one would place less trust in him as a result.

I did negatively update based on reports of colleagues saying he has a tendency to be deceptive, though.

Pure psychosis. Wtf even is a “machine god”? Read what you’re saying

Psychosis has a meaning, use a different term.

Delusion, then

Imagine for a minute that someone creates an intelligence capable of broad, deep, and speedy perpetual self improvement. What sort of relationship do you have to that entity after a month? A year? A hundred years? A thousand?

You mean like human beings?

No.

Do you know a human being capable of continual self improvement for a thousand years?

not yet, but we do know of some humans who are pouring billions of dollars into making that a reality, eg. the hot mic between Xi and Putin as well as some valley folks who are proud to announce their ambition on this matter

Well that’s called society or civilisation

Bizarre there are still denialists in places like this.

(It'll probably be masses of demigods, not one Singleton God, but obviously it's going to be some absurd frightening situation akin to this within our lifetime assuming no global AI ban.)

> Bizarre there are still denialists in places like this.

“Places like this”? When did this site become a place of worship and blind faith?

It's not a certainty by any means, but to act flabbergasted at the thought of the existence of ASI by 2070, in 2026, on a tech site, is just silly.

Wow, your delusion even has a whole fan fiction lore

Let's check in in 50 years and see which of us was right.

That’s irrelevant really. Even if it might happen it still doesn’t justify your current thinking. It’s contingent. read some philosophy plz

Trustworthiness arguments aside:

You're saying you think there might be machine demigods in 50 years, but that one shouldn't use the pace of AI advances today to update positively on the probability of that?

Reply on news.ycombinator.com