0: https://embracethered.com/blog/posts/2024/hiding-and-finding...
0: https://embracethered.com/blog/posts/2024/hiding-and-finding...
So stupid, the fact that we can't distinguish between data and instructions and do the same mistakes decades later..
We need AI because everyone is using AI, and without AI we won't have AI! Security is a small price to pay for AI, right? And besides, we can just have AI do the security.
But, even with those tokens, fundamentally these models are not "intelligent" enough to fully distinguish when they are operating on user input vs. system input.
In a traditional program, you can configure the program such that user input can only affect a subset of program state - for example, when processing a quoted string, the parser will only ever append to the current string, rather than creating new expressions. However, with LLMs, user input and system input is all mixed together, such that "user" and "system" input can both affect all parts of the system's overall state. This means that user input can eventually push the overall state in a direction which violates a security boundary, simply because it is possible to affect that state.
What's needed isn't "sudo tokens", it's a fundamental rethinking of the architecture in a way that guarantees that certain aspects of reasoning or behaviour cannot be altered by user input at all. That's such a large change that the result would no longer be an LLM, but something new entirely.
I've tried to think of a way to solve this at training time but it seems really hard. I'm sure research into the topic is ongoing though.
You are in manual breathing mode.
I think this will be something that's going to be around a long while and take third party watching systems, much like we have to do with people.
But then, security is not a feature, it's a cost. So long as the AI companies can keep upselling and avoid accountability for failures of AI, the stock will continue to go up, taking electricity prices along with it, and isn't that ultimately the only thing that matters? /s
Hire a consultant who can say you're following "industry standards"?
Don't consider secure-by-design applications, keep your full-featured piece of jump but work really hard to plug holes, ideally by paying a third party or better getting your customers to pay ("anti-virus software").
Buy "security as product" software allow with system admin software and when you get a supply chain attack, complain?