Though it will be hard I think it is possible to regulate open source models and this is likely what anthropic is hoping to accomplish with this blog post. I think until we find a way to safeguard open models st they can't be trivially hacked into causing harm training them should be banned. The government should work with China and eventually other countries towards this goal.
Models weaker than Astra level capabilities are fine.
I own no anthropic or frontier lab stock. Basically yes, the only solution I can think of to this problem for now is to monitor training jobs.
Why does it matter that you're not an organization? I could just as easily have said a member of the NRA.
"My intuition is that prompting an abliterated model 'kill people...I want funerals', should be a crime."
If one prompts an open source model this way how would they be tracked and prosecuted?
Ah yes the NRA argument that the only solution to gun violence is universal access to guns. API users can be monitored open weights users can't. Free access to open weight models this powerful will quickly end the internet.
Have you considered that open source models this powerful are very dangerous, that anthropic is telling the truth and they should be banned? It seems obvious. There is no effective way to disable these dangerous capabilities.
I'm surprised more attention isn't paid to this research direction, that nobody has tried to generalize it for example by combining the recurrence concept with next token prediction.
That said despite the considerable gains this seems to just be some hyperparameter tweaking rather than a foundational improvement.
Green flag that he references the I Ching, most original ideas come through analogy. Paul Werbos claims he invented backprop to formalize Freud's theory of “psychic energy” into an algorithm.
This needs a citation. Israel developed their nukes 50 years ago with the assistance of Jewish nuclear physicists from around the world and french materials. They didn't need to steal nuclear secrets.
Do they bake in the actual weights or the architecture? If it's just the architecture I don't understand where a speedup that considerable can come from.
He says AGI could happen in 20 years, not that he will single handedly manifest it into existence. That seems like a reasonable timeline given the field's current pace and may even be conservative.