Hacker News Reader

Comment by woadwarrior01 | Hacker News Reader

Isn't this the same as Simon Willison's LLM cliche highlighter?

https://tools.simonwillison.net/llm-cliche-highlighter

Never seen that tool before, but it has an obvious bug: paste in markdown-formatted text with `-` bullet points, and it told me a sign of AI writing was: `Repeated sentence openers · 3 sentences opening “-”`

Now, the text I pasted in so happened to really be AI-generated, but I don't think using markdown bulleted lists is a reliable sign of AI-generation.

A coworker assumed I shared AI slop because I wrote about 200-300 words of text with nice formatting: different heading text levels, monospace formatting for variable names, used different highlight colours, etc.

Can’t even write pleasant technical documentation by hand after memorizing hotkeys without being falsely accused…

Interesting! I tried it with unordered Markdown lists (both nested and single-level) and I couldn't reproduce it. Would you mind sharing a small repro? In some ways I guess the brittleness is to be expected since IIUC, it uses regexes under the hood.

Here is my input: https://gist.github.com/skissane/dfb7a569968224ec98cda757371...

To be clear, the issue is not that it says that text is AI-generated – that text is indeed AI-generated. The issue is the detection reason it gives:

Repeated sentence openers · 3 sentences opening “-”

A nested markdown list is not a reliable sign of AI-generation, humans were using them long before any AI was.

Similarly, its second "AI tell" is:

Echoing sentence runs · 2 sentences echoing “google sso auto create”

Two sentences beginning with the `GOOGLE_SSO_AUTO_CREATE_` is not a sign of AI generation, it is a sign of quoting code

Only the third match is a genuine "sign of AI generation"

A reliable sign of AI writing is the middot in the tool's output. But that is to be expected, given the tool's author.

Reply on news.ycombinator.com