393 karma · joined June 29, 2026
All you need is the untrusted content (or more generally vulnerable content since how you feel about the content doesn't matter) leg and something you care about to be attacked.
E.g. could be as simple as a prompt injection that causes your LLM to output a prompt injection that then gets inserted somewhere else.
Or without exfiltration rm -rf /
Or a social engineering attack.
So there are other bifectas and trifectas.
Yeah there may be biases but there are imperial biases in western models too.
There is some element of responsibility on the user to guide and monitor the model/harness and not let it rip to burn tokens.
I think Zed doesn't need to build this though.
Yeah this attack is possible on all modern agentic systems.
* Access to your private data
* Exposure to untrusted content
* The ability to externally communicate in a way that could be used to steal your data
(https://simonw.substack.com/p/the-lethal-trifecta-for-ai-age...)
And blocking it wholesale reduces usefulness of the agent so it is a tradeoff.
I love this question because of the diverse answers given and we learn from each other.
I am guessing keeping them all in every request is cache efficient though.
I'd also look at willingness to pay in the analysis.
Also I love AI sites. Fancy font, plain serious style, we "introduce" rather than "release". It's an AI not an animal after all.