Hacker News Reader

Comment by chris_money202 | Hacker News Reader

Unfortunately if it was from a compromised extension this is going to be more justification for creating closed environments like what Google is doing with android and Apple has already done with iPhone.

View on HN

Why not simply have both? This does not have to be an either-or decision. Have a default repository with vetted extensions, but leave the option to install from other sources open.

Enterprise will always choose the less risky option so if there is either-or its vetted extensions only.

For consumer it's kind of already like this in a way, there are "verified" extension providers.

Overall, I think this is just going to lead to a lot more scrutiny. I'm sure one of the first things asked when this was discovered was how can it be prevented and I'm sure one of the first answers was get VsCode to lock down extensions. Enterprises love the easy answers

In the age of LLMs, vetting can even be done in a CI/CD. What's the big deal?

Token use? Non-deterministic outcomes?

Whatever findings they come up with, must be formalized to avoid non-determinism.

Draft an integration test of every finding.

The malicious extension makes calls to haxx0r.net? Draft a case in your integration test that intercepts this.

But why will LLMs get this right when Web-of-Trust and blacklists didn’t? For a long time we’ve had different heuristics to detect abuse, and it’s always been a losing battle.

E.g. an extension that sends requests to an IP. Do you block all network access? IP ranges? Well, we’ve had firewalls for ages, hackers still craft successful vectors.

I'd say it's different this time, because LLM can form and substantiate opinions of their own, on their own. You could actually assume you have the full source code of any binary for it to analyze, since in theory everything is decompilable, and you could then also diff between releases to save 99% tokens.

Think this is more an argument against them in security than for them. You don't want the LLM forming the opinion that something is secure when it is not because it has "microsoft" in the name of a binary or something else. There are a lot of ways to mis-direct their reasoning and it's almost impossible to control that 100%. A lot of AI providers are learning from the mistakes, so implies something is going to get hacked once in some unique way and then LLMs won't make the mistake again with 99% certainty.

Let's strip all strings then before analysis? to prevent prompt injections and such.

Reply on news.ycombinator.com