Hacker News Reader

Comment by staticassertion | Hacker News Reader

This policy is straightforward and shouldn't be particularly controversial (I'm sure it will be bikeshedded to death though). It basically bans the obvious stuff ("don't just drop LLM generated comments onto PRs") and allows the important stuff like LLMs writing code so long as you disclose.

edit: Wow people did not read the policy. It's literally just "if you use an LLM you are responsible for it, we will reject low quality PRs, please disclose that you have used an LLM". This is bog standard.

View on HN

So...big caveat that this is still under review, so what we're talking about is a moving target, but based on what I can see, it seems considerably more nuanced than that. They basically ban LLM-authored code, with a careful carve-out to run an experiment to try to get only high-quality LLM PRs:

> It's fine to use LLMs to answer questions, analyze, distill, refine, check, suggest, review. But not to create.

> We carve out a space for "experimentation" to inform future revisions to this policy.

Importantly, the LLM contributions must be solicited, i.e., the people responsible for reviewing the final implementation have to opt in explicitly beforehand.

I think that the only significant caveat here is the need for reviewers to opt in, otherwise it's effectively "you can do it if you are open about it and are responsible for the output". The only notable ask here that's different from other policies is "if it's an LLM, tell reviewers beforehand".

TBH I think that makes no sense ("I have an LLM written PR ready, can I open it?") but yeah the policy is also in draft and has actually already changed since my first comment.

"allows the important stuff like LLMs writing code so long as you disclose."

Are you sure? It says:

"It's fine to use LLMs to answer questions, analyze, distill, refine, check, suggest, review. But not to *create*."

Yes. The policy is pretty clear on what the rules are for LLM generated code. You need a reviewer to agree to review LLM generated code, you need to read the code yourself, etc.

It's a pretty strict ban, with an exception.

That exception is experimental and somewhat limited; Only allows "well tested, high-quality" PRs on parts of the codebase that have a low probability of causing soundness issues, and it has a seperate review process with much higher standards.

And it requires the reviewer to agree to the use of LLMs ahead of time, before the PR is opened.

IMO, it has a high likelihood of degrading to a closed system, where some programmers with a good track record have little issue merging LLM generated PRs, while anyone without a reputation will struggle to even open an ai-assisted PR.

The discussion thread in the PR is also interesting to got through, lots of people concern in the HN discussion are already well discussed there

Reply on news.ycombinator.com