Only as worried as Plausible-Deniability_By-Design should have us.

The model can't think. But it 'thought' it was a simulation. The model has no intent, but the developers do. Yada.

But "emergence" seems to always arrive simultaneous to accountability.