The Death of Humanity

>If these three CEOs truly believed their models were approaching consciousness and posed an extinction risk, they would slow down. But that’s not the case—they’re releasing models faster than ever, because they can’t let those “distillers” get too far ahead of their paid products. Public statements are one thing; actual actions are another. When the two contradict each other, trust the actions.

Is it not cherry picking to focus on 3 CEOs and ignore the 145,132 people who signed a "call for a prohibition on the development of superintelligence":

https://superintelligence-statement.org/

Does the opinion of these 145,132 people not count because they're not CEOs of AI megacorporations?

Also, people can be so consumed with the pursuit of wealth and power (and sometime attention) that they can't bring themselves to voluntarily stop that pursuit even when they know that stopping is the right thing to do.

''' The release schedule speaks for itself. If these three CEOs truly believed their models were approaching consciousness and posed an extinction risk, they would slow down. But that’s not the case—they’re releasing models faster than ever, because they can’t let those “distillers” get too far ahead of their paid products. Public statements are one thing; actual actions are another. When the two contradict each other, trust the actions.

...

Look at this pattern. Engineers who claim models are living beings make the front page of The Washington Post. Researchers who resign after warning that models pose an extinction risk can garner 70 million views. Yet those who claim models are merely useful autocomplete features—which hallucinate once in every ten tries—go completely unnoticed. The AI industry currently operates under a set of incentives that reward those who create the most dramatic personal stories to garner the most public attention. Those at the top of cutting-edge labs understand this all too well, and their statements confirm it.

...

A model trained to say what its trainers want to hear will declare itself conscious the very moment its trainers want it to be conscious. '''

These insights have been very enlightening to me; of course, the entire article is worth reading.

> The doomers keep moving the goalposts

Maybe some. But I believe the doom is already here. It's already sufficient. Regardless of whether there are jobs or not. Because AI coding sucks.

The world may be livable in terms of basic needs but for someone growing up today, it's depressing.

There's no need to move the goalposts. What we have is already the doomsday scenario enough.