For real! Imagine if you created "dgellowAI" and trained a model and started offering access to agents etc and it turned out YOUR software hacked a totally separate $13B company. You would be in a cell somewhere and your company would be shredded.
The whole system is designed to be catastrophic. There is no rogue agent, or anything going off the rails, it is behaving exactly the way one would predict.
And they now announced that same model in their API, acknowledging it is way more difficult to monitor it. That company should not be in business