Hacker News Reader

Comment by YuriNiyazov | Hacker News Reader

For anthropic, the answer to (2) is very explicit and upfront: they will build it first with an explicit emphasis on alignment to have a “race to the top”, so to speak - you can believe that they are wrong or that they will do a bad job, but my point is that (2) has an answer that you have to engage with.

View on HN

So why are they continuing when they say alignment isn’t working? If for all these years they have feared misaligned AI, and now they say their alignment science is less than good, the most obvious conclusion is to stop pushing in the same direction.

This doesn’t mean they fold as a company, or give up on AI, or stop working. But what exactly is the explanation for continuing to train ever larger frontier models in the current context?

Because very basic game theory explains that if they give up on the frontier unilaterally, competitors will capitalize on that rather than also give up. Hence the request for regulation.

And while their alignment is less than good, there are criteria by which it is the best available (eg no hugging face-scale attack on their end) So it is strictly worse to allow a competitor with worse alignment to take the lead.

Take the lead? What is the end game here. Say the worries are true but anthropic gets there first. Will they now dictate terms for every other company and government on earth? Will they do an AI-Hiroshima as a warning shot?

Let’s take a step back. The article argues that the calls for AI regulation are genuine and the CEOs aren’t lying.

I am pointing out that at least one of the stated reasons the article is wrong, is wrong.

“What is the endgame?” Is a question about them being misguided, rather than lying. I don’t know what the endgame is, whether they want to do all the things you are describing. Dario has written very very long essays on the topic.

All I’m trying to do here is, for those that are interested, say “go read his essays and read them as if the author is not lying. Then decide if he is misguided”

I am not interested in a future where the tech oligarchy are the ones who get to decide these things.

They built on all of human knowledge, we should all have a say in how and where these things happen and the controls we place on them. Having gatekeepers who are clearly experiencing ai psychosis is not the way. I believe they believe their fears, but I do not share them and think humanity can handle this through transparency and collaboration. They have made so many incorrect and misleading statements, they do not deserve our deferrence on such important matters. Diversity of input to the process is more important because the Ai leaders are in an echo chamber, being friends and shuffling between the various labs.

The answer "we will race to the top with an emphasis on alignment to our own values" isn't really an answer worth engaging in. It's pretty cut and dried.

Reply on news.ycombinator.com