6,194 karma · joined March 18, 2015
Any opinions expressed are purely my own and not necessarily those of any employer, past, present, or future.
Personal Site - https://www.mccune.org.uk Blog - https://raesene.github.io Mastodon - @raesene@infosec.exchange Blue Sky - @mccune.org.uk
You could still overload with too many skills but it helps at least.
There's a load of ways that a repository owner can get an LLM agent to execute code on user's machines so not a good plan to let them run on your main laptop/desktop.
Personally my approach has been put all my agents in a dedicated VM and then provide them a scratch test server with nothing on it, when they need to do something that requires bare metal.
There's not a tonne of tooling for that use case now, although it's not too hard to put together I vibe-coded something that works for my use case fairly quickly (CC + Opus 4.5 seemed to understand what's needed)
My feeling is that for many/most students, getting a great understanding of the course material isn't the primary goal, passing the course so they can get a good job is the primary goal. For this group using LLMs makes a lot of sense.
I know when I was a student doing a course I was not particularly interested in because my parents/school told me that was the right thing to do, if LLMs had been around, I absolutely would have used them :).
For example if you just ask an LLM in a browser with no tool use to "find a vulnerability in this program", it'll likely give you something but it is very likely to be hallucinated or irrelevant.
However if you use the same LLM model via an agent, and provide it with concrete guidance on how to test its success, and the environment needed to prove that success, you are much more likely to get a good result.
It's like with Claude code, if you don't provide a test environment it will often make mistakes in the coding and tell you all is well, but if you provide a testing loop it'll iterate till it actually works.
There are valid criticisms of Docker/Podman isolation but it's not a binary "secure/not secure" thing, and honestly in this use case I don't see a major difference, apart from it being easier for a user to weaken the isolation provided by the container engine.
Docker/Podman security is essentially Linux security, it just uses namespaces+cgroups+capabilities+apparmor/SELinux+seccomp filters. There's a larger attack surface for kernel vulns when compared to VM hypervisors, but I've not heard of an LLM trying to break out by 0-day'ing the Linux kernel as yet :)
Assuming a standard Docker/Podman container with just the project directory mounted inside it, what vectors are you expecting the LLM to use to break out?
Fundamentally any security mechanism which relies on users to read and intelligently respond to approval prompts is doomed to fail over time, even if the prompts are well designed. Approval fatigue will kick in and people will just start either clicking through without reading, or prefer systems that let them disable the warnings (just as YOLO mode is a thing in Claude code)
In the early 2000's banks were paying ~£1000-£1200/day for pentesters from boutiques and when I stopped being in that industry ~5 years ago, it was largely the same or even lower for larger companies that could negotiate day-rates down. Big-4 tried to charge more but that's really tricky when you're in direct competition with boutiques who have more testers than you.
By contrast US rates were a lot higher ($2k+/day) and also scopes were larger. A UK test for a web app could be as low as 3 days (even less for unauthenticated) where the US tended to be 1-2 weeks.
One reason they've gone down is outsourcing to lower cost regions, and I'd guess that LLM/AI automation will accelerate that trend...
"I decided to make up for my dereliction of duties by building her another app for her sign business that would make her life just a bit more delightful - and eliminate two other apps she is currently paying for"
OP used Opus to re-write existing applications that his wife was paying for. So now any time you make a commercial app and try to sell it, you're up against everyone with access to Opus or similar tooling who can replicate your application, exactly to their own specifications.
The question I'd be interested in is, outside of markets where there's a lot of compliance requirements, how much demand is there for this as a paid service...
People like lower CVE images, but are they willing to pay for them. I guess that's an advantage for Docker's offering. If it's free there is less friction to trying it out compared to a commercial offering.
I saw at least one large company that migrated from Notes to exchange and they got the email/calendaring bit done quite easily and were still running notes servers for line of business applications years later.
In a fully authoritarian state of course you likely don't have to worry too much about proof, but I'd suggest the UK has a ways to go for that.
On the piracy front, well we've seen how successful they were in stopping piracy.... not at all.
If they're going down that route I'd expect the first service to be banned will be Tor, I'm actually mildly surprised they haven't tried that already.
While we still have alternate operating systems, that won't be a universal control of course. You'd have to stop people owning general purpose computing devices for that to be fully effective.
For the UK I'm kind of doubting they'll put enough money into it to make it good, so we'll get the ineffective version and politicians will get stories like this one written about their efforts.
Then you're into "what about all TLS connections" which can be used to send traffic, so you have to do TLS interception at scale, which is a very non-trivial problem to try and solve.
Then you're into non-TLS encrypted protocols, so your only option there is to block anything you can't intercept.....
At that point you've pretty much broken Internet access in your country, might as well just chop the cables :P
Trying to restrict access to content on the Internet by requiring "robust" age verification was never going to achieve the goals they stated, and has a number of predictable (and already seen) negative side-effects.
Unfortunately governments all over the place seem intent on continuing this type of regulation, I presume so they can be seen to be doing something. Good time to be in the VPN game, I'd guess.
The goal of the capabilities system was to allow processes and users to gain a small portion of root privileges without giving them all.
In the "old days" ping on a Linux host would be setuid root, so it essentially had all of root's rights. In more modern setups it either has CAP_NET_RAW or the ping_group sysctl is used to allow non-root users to use it.
1. You can just add the capability CAP_NET_RAW to your process, at which point it can ping freely
2. There's a sysctl that allows for unprivileged ping "net.ipv4.ping_group_range" which can be used at the host level to allow different groups to use ICMP ping.
Use a container or VM, place the code you're working on in the container or VM and run the agent there.
Between the risk of the agent doing things like what happened here, and the risk of working on a malicious repository causing your device to be compromised, it seems like a bad plan to give them access to any more than necessary.
Of course this still risks losing things like the code you're working on, but decent git practices help to mitigate that risk.