It would work no different from the current apps on the backend; (chat)gpt will fill a typed object, for instance a Transaction, software will validate this object as it does now (you can fill an object in an illegal way now and send it to the api; no difference), show it in the chat window as a strict UI to ask you if you are sure and then execute. The gpt part is only for you, not for the backend.

So people can try to make it do whatever; the banking backend logic won’t execute it, so it doesn’t matter. The user will just be wasting their own time.

Until of course neo-neo-web3-gpt-banks pop up that are ‘built up from the ground with ai’. But those will fail regulatory before they launch anyway.

Yeah, that was my first thought. Adding a layer of UI with documented unintended behavior is a liability landmine.