Do that automatically for all code downloaded from the web and run outside a sandbox.
Maybe won't catch everything, but should catch most evil stuff, especially if a variety of models and prompts are used.
Why is the answer for Javascript developers "don't use leftpad" but for the AI crew it's "convert the source code to tokens and attempt to strip out the bad stuff without breaking the rest"?
b) You can have the LLM use separate sub agents for different files/ code.
c) You can have the LLM do analysis using grep and other deterministic tools ex: "use grep to find 'unsafe' calls"
2. We have a long history of using heuristic technologies to detect attacks. We can infer that other heuristic technologies can be combined in a successful manner.
3. Shortcomings of LLMs are directly addressed by removing attacker controlled information from the input, which I specifically called out (using tools like grep for pattern matching + using sub agents to isolate contexts). This has been demonstrated already in a number of ways - feeding the LLM derived facts instead of attacker controlled data is the well worn path to avoiding injection attacks.
I stopped reading after that.
I guess maybe you've learned a new word today? Hope so.