These are relatively brand new systems that process an incredible amount of data to form their "world view". The word non-deterministic gets thrown around a lot, but you have to understand that there's absolutely nothing deterministic about these architectures. The only way to know that certain qualities could emerge is to observe the qualities emerging.
Giving an agent instructions that, for example, instruct it to only perform GET requests, and then observing that the agent does not always respect that request, is not a failure of security or configuration: it's data being gathered.
Yeah, we're going to become more diligent about the protections that we put in place beyond any level of protection that we've ever employed before. You can say that we have decades of security research and experience, but we're watching all of that fall more and more day by day, finally putting a real delta on how secure we thought we were versus how secure we actually are [1]. Additionally, adversaries have never existed inside of the systems that we hoped to secure in the first place.
So go ahead, enumerate all of the ways in which you think that you can lock down these systems, but do realize that nobody has actually solved this problem adequately yet.