Hacker News Reader

Comment by polyrand | Hacker News Reader

Instead of containers, which may not always be available, I'm experimenting with having control over the shell to whitelist the commands that the LLM can run [0]. Similar to an allow list, but configured outside the terminal agent. Also trying to make it easy to use the same technique in macOS and Linux

[0]: https://ricardoanderegg.com/posts/control-shell-permissions-...

View on HN

Not specific to LLM stuff, but I've lately been using bubblewrap more and more to isolate bits of software that are somewhat more sketchy (NPM stuff, binaries downloaded from GitHub, honestly most things not distro-packaged). It was a little rocky start out with, but it is nice knowing that a random binary can't snoop on and exfiltrate e.g. my shell history.

You might might my (alpha-level) attempt at this: https://gitlab.com/txlab/ai/sandcastle

Looks like it's probably neat, but is kinda inverse from what I myself want. I want:

- something general-purpose (not specific to LLMS (I myself don't use agents--just duck.ai when I want to ask an LLM a question)) - something focused on sandboxing (bells and whistles like git and nix integration sound like things I'd want to use orthogonal tools for)

I really like this and we're doing a similar approach but instead using Claude Code hooks. What's really nice about this style of whitelisting is that you can provide context on what to do instead; Let's say if `terraform apply` is banned, you can tell it why and instruct it to only do `terraform plan`. Has been working amazing for me.

Me too! I also have a bunch of hooks in claude code for this. But codex doesn't have a hooks feature as polished as claude code (same for their command permissions, it's worse than Claude Code as of today). That's why I explored this "workaround" with bash itself.

An interesting exercise would be to let a friend into this restricted shell, with a prize for breaking out and running rm -rf / --no-preserve-root. Then you know to switch to something higher-security once LLM capabilities reach the level of that friend.

You have to put them in the same ACL, chroot, whatever permission context for authorization you’d apply to any other user human or otherwise. For some resources it’s cumbersome to setup but anything else is a hope and a prayer.

This is how I've been using Gemini CLI. It has no permissions by default, whether it wants to search google, run tests, or update a markdown file it has to propose exactly what it needs to do next and I approve it. Often its helpful even just to redirect the LLM, if it starts going down the wrong path I catch it early rather than 20 steps down that road.

I have no way of really guaranteeing that it will do exactly what it proposed and nothing more, but so far I haven't seen it deviate from a command I approved.

I didn’t check, but sometimes Claude Code writes scripts and run them (their decision); does your approach guard against that?

It depends. If you allow running any of bash/ruby/python3/perl, etc. and also allow Claude to create and edit files without permission, then it won't protect against the pattern you describe.

Reply on news.ycombinator.com