Isn't this basically a lot of hand waving that ends up being isomorphic to SQL injection?
Thats what we're talking about? A bunch of systems cobbled together where one could SQL inject at any point and there's basically zero observability?
Thats what we're talking about? A bunch of systems cobbled together where one could SQL inject at any point and there's basically zero observability?
Therefore it's possible to prompt inject and tool inject. So you could for example prompt inject to get a model to call your tool which then does an injection to get the user to run some untrustworthy code of your own devising.
[1] See the excellent series by Simon Willison on this https://simonwillison.net/series/prompt-injection/