Hacker News Reader

Comment by sebstefan | Hacker News Reader

Say you have an expert in a field who's not stellar at english, someone like me, and I use a large language model to generate a response on a subject I'm an expert in? That's not harmful. I'm not blindly pasting the output to their answer form, my point is still to post a correct answer. I'd obviously double-check. It would be able to produce a better-worded response than my semi-bilingual brain ever could so there is added value there, without even accounting for the time it saves me. I edited this post like 4 times to add an S somewhere or remove one already. Blanket bans on LLMs like the moderators want to do are lazy. The criteria that should matter is whether you answer the question properly or not

Then the second point is, on a blind sample of question answers, how well can they tell if something has been generated? I bet it's not stellar.

I hope Stack Exchange stays on their position

View on HN

But that is the point, if you use AI to improve what you would answer, that is totally fine, but not that you just copy and paste what it says.

And, I guess they will just realise it is AI generated when someone notices that what is written does not make any sense, but it is still really well written. In any case, very difficult to identify, for sure.

Most people can read ESL English better than GPT's "wordy advertisement" style English.

If everyone used LLMs like that, there would be no problem. The thing is, few do, and with the new policy, moderators are not in a position to do anything about it.

The thing that I also don't get, is that I haven't seen an example of that being an issue on stackexchange yet

I also don't see the end goal making a bot that answers questions on SE. It doesn't make money. Maybe to get points? But once you're past their thresholds there's no reason to keep doing it, and you get there quick. Accounts don't get sold to advertisers like on reddit. You'd at most do it once. And who would even do that? The very narrow niche of people who'd want to boost their SE points? Maybe you're shooting for the leaderboard but if that's the case... you'd get noticed, wouldn't you?

I could see some kids doing it, if that's the big threat they're facing... then they're over-reacting and it still doesn't warrant blanket bans on LLMs

> I also don't see the end goal making a bot that answers questions on SE.

A tip I learned long ago: Never ask a geek "why?", just nod your head and back away slowly.

Why not? If an answer is wrong it's wrong. Why does a GPT ban prevent removing wrong answers?

What a GPT ban does do is give moderators an extremely subjective tool for removing answers. The GPT detectors are unreliable, so that leaves the moderators doing a gut check on whether this particular case is a false positive.

Instead, why don't they just use the GPT detectors to detect answers that might be wrong and then moderate based on existing policies?

In short: moderators do not remove answers based on (their perception of) technical merit, non-moderators cannot remove answers as long as the score is non-negative, and GPT drivel sounds convincing enough to get upvotes from credulous newbies, so the answers have positive score.

So fix the first one. Provide a process for removing answers that are demonstrably false. Convincing-sounding drivel that gets upvotes isn't a new problem on SO, it's just amplified by the new tech.

But this is a problem of scale. If I wrote a bot that posted 10k AI generated questions and answers per hour, you would surely agree that it would be unreasonable to require every answer to be human-reviewed before it can be removed - it would be an incredible waste of time and effort. (Also note that filling SO with LLM-generated content is entirely pointless - someone who wants LLM-generated answers can just go ask an LLM).

Now spread that one bot over hundreds of users and you have the same end result. That's why the communities of different SE sites all ended up with a similar policy.

I get that, but the problem of scale is one that you have no matter what. Either you have to detect and filter out AI posts at scale, or you have to detect and filter out bad content at scale.

I'm not suggesting either problem is easy, but taking the "ban AI" approach is harmful for several reasons. Detecting AI without false positives is extremely hard and puts too much decision-making power in the gut reactions of moderators. Additionally, banning it has the negative side effect of making the tool unavailable to people who might legitimately benefit from it when producing good content, such as ESL speakers using it to polish their English.

We need to moderate based on the effect of the user's content on the community, not the technology used to produce the content. If you're dealing with a bot posting 10k questions and answers per hour, it wouldn't matter if that bot were using GPT or just re-posting content scraped from the web—the abuse isn't in the use of AI, it's in the spam. So make a rule against spam and auto-ban people who do it.

I said this elsewhere in the thread too, but the ban was always meant to be temporary: https://meta.stackoverflow.com/questions/421831/temporary-po... The moderators recognize it's inevitable and wanted to find a way to helpfully integrate AI. The strike is not about making the ban permanent; it's about being suddenly overruled in secret, cutting off the possibility for dialogue to find a good solution.

Exactly, how would anyone know I answered with the help of an AI? I got bored reading the post drone on about banning GPT. Weird. None of it resonated with me.

The article talks about GPT detectors. They don't work amazing, though.

Having flagged hundreds of such answers before SE suppressed LLM moderation, I can vouch that they stick out like a sore thumb:

- they're all written in the same style ("It looks like you're trying to do convert a string to an integer...") - they have perfect grammar - they're often/usually wrong - they tend to be tone-deaf to the question or are answering a different question than asked - their formatting is often poor by virtue of copy-paste from ChatGPT and the author's lack of familiarity with markdown - they're usually posted by new or new-ish users that have few reputation points (ostensibly, their goal is to farm reputation rather than help curate a quality resource) - they're posted in large quantities (a dozen or more in an hour) that'd be virtually impossible for a human to churn out (especially a brand new user!) - they're posted in random tags that show no clear connection from one to the next--most answerers are subject matter enthusiasts or experts and stick to a narrow range of tags - they tend to contain hallucinations obvious to SMEs, like invoking methods that don't exist - the code is plain wrong when executed - the code style is "boring"/"vanilla" and tends to steer clear of idiomatic language features that a seasoned programmer would employ and has few formatting quirks that a human might use - the code is often heavily commented in a predictable and artificial manner - the explanation after the code and overall layout/flow of the post is often the same - they tend to be dramatically different than the user's normal answers, which have typos.

As you'd expect, it's difficult to _prove_ that a particular answer was generated by an LLM (the fact that LLMs can't reliably detect themselves is part of the problem--they're inaccurate!). However, the possibility of occasional false positives (SE has provided no actual evidence of this being an issue), seems a necessary price to pay, and could be solved in a more balanced manner than prohibiting all LLM moderation. SO would be unusable if it became a stale cache of mostly-incorrect LLM spew, which is what SE's new moderator guidelines seem to be OK with.

If I want an LLM answer, I'll ask an LLM. I can then prompt engineer and iterate. If I want a human answer, I want to be able to ask on SO so I can be guaranteed I'm "speaking with a human".

Disclosure/context: I'm a power-user on strike.

Reply on news.ycombinator.com