Anthropic says a key AI safety alarm has been triggered, days before its IPO

These people are not in charge of their faculties; havne't developed the intelligence to rationalize their own actions; and lastly, are high on their own supplies.

That said, these models are as powerful as the security systems you bypass to test them. So the danger is more this is a private company well funded and could hold people hostage with demands.

> This one is triggered when AI is doing enough of Anthropic's research to roughly double how fast its models improve. The policy says that pace would pack two years' worth of AI progress, at its 2018-2024 rate, into a single year. The filing says AI "has already been accelerating our research and development." Crossing the line means more safeguards, not a shutdown.