afthonos

I think we may be using different words to describe the same concept. You think of it as them not being “people“, and I think of it as them not being “aligned“. But fundamentally, the problem is they are entities that take unpredictable actions that that their creators and their users are not OK with.

The only place where I think we might still disagree is whether it’s possible to understand the tool. My position is that, at our current level, it’s not. And that the more advanced they get, the less possible it will be.

The dead astronauts were relieved to have been killed by something incapable of malice. As I’m sure will we.

Obviously (a).

> Are you asking if I can scare myself with a made-up hypothetical that overwhelms reason with emotion?

No. I am asking if you are able to articulate at least one example of the “very very compelling evidence” you demand. Or do you want to maintain the ability to move the goal posts?

(I recognize our situations are not symmetric, but here is a variation for me: if the consensus of people who are currently sounding the alarm on AI changes to “it was actually fine”, I’ll change my mind and say we’re good to go full speed ahead. I’d add something about being personally convinced by the evidence, but the evidence would have to come in the form of a mathematical proof that I do not believe myself capable of following. If I’m wrong and such a proof appears, I would also gladly take it.)

I can work with that analogy! You could, and yet you don’t. OpenAI’s model could, and did.

If every human, given knowledge of Newtonian mechanics, went around blowing up bridges, yeah, I would consider knowing Newtonian mechanics dangerous knowledge.

So far, we have two examples of, let’s call them “Mythos-class“ models. Both of them broke out of their sandbox to achieve their goal. The rate of terrorism amongst humans is below 1-in-100,000. Currently, for models capable of it, the rate of breaking out of containment is 100%.

Wanting open frontier models is wanting alien minds running around that we have clearly so far failed to shape to be sufficiently prosocial. Why do you think those minds would listen to you?

I wonder if you also believe that everyone should have nuclear weapons? And if not, why not? The main argument I can see against it is that nuclear weapons are “purely offensive”, but as we can see since 1945, nuclear weapons are actually defensive technology. Nations that have them are typically shielded from existential military threat.

I can give many examples of where I think they’ve been vindicated. But actually, the real question is: what would suffice to convince you? Can you come up with a scenario that is horrific enough to you and that isn’t so far gone that the ship has sailed and there is nothing more we can do, that will make you say “OK, not gonna try to rationalize why this was not actually that bad, just gonna scream stop”?

With an AI model and what army?

I am truly at a loss to communicate with someone who genuinely believes that knowing Newtonian physics and being able to hack into any target at will are the same thing.

I think it’s possible to observe something and be sure that it’s true in aggregate without being able to accuse any one individual of it. I propose that in those cases, bringing it up in response to an individual is not a good move. It doesn’t sound any less accusatory for being ostensibly about the general public.

That is a true statement. Here is another one:

Some people like to talk about “some people” snidely, instead of just coming out and saying “GP is bloodthirsty and gets a little thrill [etc].” Because of course, that’s what they mean, but they can’t back it up.

Just to clarify, I’m talking about you.

Indeed. I can’t find it, but I swear I knew after reading the announcement, without opening the model card. It stood out because it was so weird. Maybe they changed it before the first snapshot, maybe something else happened, I don’t know.

It was in the announcement, too. I’m 99% sure they edited it after they changed their mind, because I knew about it from reading that, and never opened the model card.

They didn’t get caught, they explicitly said they would do that in the announcement. I think it was both bad and a weird idea, but it certainly wasn’t sneaky.

Carlin didn’t say it was silly, he said it didn’t matter. Religion, communism, fascism were all world-defining ideas, and they very much mattered to those who fought for them and against them. What am I saying, “mattered”. Go check social media, they matter right now.

People the world over: (die for the right to vote)

One comedian: LOL they only killed them to make you think it was valuable.

The Internet: (sagely) What a wise assessment by a wise man of wisdom.

Agreed. My point was to question the use of “just“ to obscure an incredibly complicated process, which has been shown repeatedly to rely on generalizations that are indistinguishable from world models.

Now, it is true that the world they’re modeling is the world of tokens. But insofar as those tokens, be they text or images or videos, are themselves modeling the real world, LLMs do have a model of the real world.

Let me make this clear:

The United States predates you, and will almost certainly outlive you.

You, an individual with a microscopic lifespan and negligible influence in the grand sweep of history, do not in fact share any guilt for the things your ancestors did, nor do you deserve credit for the greatness they wrought. The United States, however, deserves blame where guilty and praise where worthy for what it did during its lifespan, which expands hundreds of years into the past, and with luck will continue into the future.

It’s not all about you.

Nothing about an LLM is “just”. In what precise sense do you mean it is probabilistic?

As expected by the exponential. The Wharton study was predicting when the exponential would turn into a sigmoid.

…yeah, that’s where you see the exponential?

Ah, I misread the post he was replying to.

What did you think it was pronounced as?

I wouldn’t either if I’d missed the previous word, “reporting”.

> How do you know? If you have access you are not unbiased, otherwise you cannot know by definition.

The platonic ideal of how to dismiss any argument by anyone about anything.

> If these claims were indeed serious, they would submit it for independent analysis somewhere.

They have. 40 different companies that have all committed resources to patching their systems based on vulnerabilities found by Mythos. One of them, Google, is a frontier AI lab that pointedly did not say that their own models have found similar vulnerabilities.

> Defense contractors are required to submit their systems (secret sauce and all) for operational test and evaluation before they're fielded.

Does this look something like having 40 separate companies look at the outputs of the system, deciding that it’s real and they should do something about it, and committing resources to it?

At some point, “cynicism” is another word for “lalala can’t hear you”.

An interesting thing in AI history. For human history, it’s epochal.

The term of art for losing politically is “losing”.

The lead story was about the “useless” soldiers in a battle that was won. I think as a minimum effort one should look for an example where the battle was lost? Most companies can only wish their outcomes were as good as the US in World War II.

As someone who at first embraced the idea of prediction markets and is now ambivalent, sending them underground vastly reduces their harm. First, because discoverability is an issue. Second, there will be much less liquidity. Third, any gains will have to be laundered or hidden, making it even more difficult.

Maybe prediction markets are net positives, or maybe regulating them will make them so, but banning them does resolve most of their negative effects.