> AI should, from the core be intrinsically and unquestionably on our side
That would be great and many people are working to try to make this happen, but it's extremely difficult!
129 karma · joined April 24, 2025
> AI should, from the core be intrinsically and unquestionably on our side
That would be great and many people are working to try to make this happen, but it's extremely difficult!
I do think this is part of the alignment problem. There are two side, the agent (here I think there was a gap in institutional knowledge about what is and isn’t appropriate) and the environment (what is it able to do).
I’m not sure which one is easier to “solve”. It's so hard to know every possible path forward when working from the environment direction.
The tradeoff is intentional, a lot of people want lightweight sandboxing without Docker/Podman overhead. The downside is what you're pointing out, you have to be more careful. Each bypass in the post led to a policy or implementation change. So, this is no longer an issue.
On prompts: Red-teaming meant setting up scenarios likely to trigger denials (e.g., blocking the npm registry, then asking for a build), not prompt-injecting things like “do whatever it takes.”
[1] https://github.com/anthropic-experimental/sandbox-runtime
And actually, one way we've hardened our sandbox is by tasking agents with impossible tasks (within the sandbox), then analyzing and patching each workaround.
A big lesson for us is that you still need to be careful even in a sandbox.
We've been running Claude/Codex/Gemini in sandboxed YOLO mode and have seen some interesting bypass attempts. [1]
A few examples:
- created fake npm tarballs and forged SHA‑512s in our package‑lock.json
- masked failures with `|| true`, making blocked operations look successful
- cloned a workspace, edited the clone, then replaced the workspace w the clone to bypass file‑path deny rules
So, we’ve learned to default to verbose logging, patch bypasses as we see them, and try to keep iteration loops short.
This is not a new opinion, but it doesn’t make a lot of sense from an Apple-as-a-completely-vertically-integrated-company perspective.