Hacker News Reader

Comment by jimmydoe | Hacker News Reader

This is the reason I think. These black markets not only pool and resell subs, but also store data and sell to whoever is training.

Encryption is useful to at least stop the latter.

Ultimately same purpose as a\ ‘s trick exposed earlier, but a much nicer implementation.

View on HN

> but also store data and sell to whoever is training

I see this as an argument against using them/Chinese models all the time, but I don't get it. I totally understand wanting to keep your data private if you're using an LLM for personal chats. But coding? I'm not working for the military, I'd gladly donate my codebase to Chinese labs if that means they can keep releasing 6-months-behind level models for 100x cheaper.

(I understand why OpenAI doesn't want this and would implement protections. I'm talking about people using this as an argument for why you as an end user shouldn't use those services.)

Yeah. I don't see the problem with Chinese prompt stealing proxies, if it's just pure free choice and discount for explicitly insecure use cases, especially when the frontier providers they route to are soft-assumed to be doing something similar.

Some workplace code base are legally not supposed to be shared.

More importantly, they train on not only code but also your interactions with the model, no matter how little you value your labor, there are values in it.

When you work on proprietary code with a lot of trade secrets contained in it, on a codebase that did cost millions of dollars of man-hours to build and that holds the company's IP, you tend to be very careful where you're sending that to.

Why would anyone working on such code send it anywhere (other than, say, to AWS for hosting)?

Source: I work on such code. We don't allow devs to use (cloud-based) LLMs.

If you think your codebase is your competitive advantage or trade secret, you're in for bad time. AI coding has made code cheap it's the least valuable part of the value add for most companies. Almost any software company can easily be replicated with a few AI prompts. The real proprietary value is in the marketing distribution partnerships and data, not code.

> If you think your codebase is your competitive advantage or trade secret, you're in for bad time.

Obviously most developers know better, but you know who doesn't care what you think? The owner(s) of the business, and various other stakeholders. Try explaining this concept to them, and see if they get it :)

The people who work in codebases that are tightly guarded aren't guarded because the developers say so, of course.

I don't think any AI company meets the bar you are setting here.

IMO the biggest argument against "sharing" your code with LLM providers is that your approach (on a high level) will be available to your competitors on the next model release assuming they ask the right questions. Not sure how much it matters, different orgs have different risk profiles.

How do you know that they don't train their models or append your prompts to add backdoors, or compromise your supply chain by including evil dependencies? This seems hugely irresponsible.

> How do you know that they don't train their models or append your prompts to add backdoors, or compromise your supply chain by including evil dependencies?

I think most of these discussions aren't about irresponsible vibe-coders, as that whole thing is mostly a fun joke more than something serious. The rest of developers who use LLMs for development, review the code the agent writes, iterates and makes changes. Think more like pair-programming, than "Write me X then deploy to production".

I know Twitter makes it seem like everyone is doing vibe-coding and YOLOing podman images into production, but it's very uncommon in a serious/production environment to act like that. While a proper structure doesn't make it impossible for the LLMs to add backdoors either via dependencies or otherwise, but it sure makes it a lot harder.

Personally, LLMs are barely able to work alongside developers and not miss anything, I wouldn't be so worried about them being able to do normal work + malicious work at the same time, as they barely handle the first part properly yet.

> How do you know that they don't train their models or append your prompts to add backdoors, or compromise your supply chain by including evil dependencies?

I read the code.

That's not bullet proof. https://en.wikipedia.org/wiki/Underhanded_C_Contest

Reply on news.ycombinator.com