Hacker News Reader

Comment by koteyur | Hacker News Reader

I've asked ChatGPT if the text (I copy&pasted the paragraph about pros/cons) is generated by it, and it said yes.

See screenshot: https://i.ibb.co/NmmytNB/Screenshot-20230209-094632-Firefox....

I'm not sure the actual ChatGPT chat interface is capable of measuring perplexity/burstiness yet (unless OpenAI integrated their recent detection work into the actual ChatGPT model which I don't think is the case).

Per GPTZero on the first few thousand characters https://gptzero.me/:

Average Perplexity Score: 632.205

A document's perplexity is a measurement of the randomness of the text

Burstiness Score: 1089.841

A document's burstiness is a measurement of the variation in perplexity

Its conclusion was "Your text is most likely human written but there are some sentences with low perplexities". There was only one sentence highlighted as possibly written by AI: "However, each tool has its pros and cons."

It's also possible to prompt ChatGPT to write with high perplexity and burstiness, so also decent chance of false negatives.

Oh, interesting tool, thanks!

And so was your comment. I knew it!:

https://imgz.org/iALu6WAz/

Oh no, you exposed me!

Do you have any particular problem with the content of the article?

There is already a ton of generated spam content to the point where search is almost broken. Spam generated by a better generator trained on spam is still spam.

Also normal etiquette is to quote text that isn't yours. To me, this reflects extremely poorly on the author.

HN etiquette is to assume good faith and to not post insinuations about stuff like bot-written text. Maybe this thread reflects poorly on us?

Nope, it's good for me. :) I just shared fun fact (that chatgpt thinks it's generated by it), but it doesn't mean that it's true or the article is bad.

What’s the accuracy of this tool?

Good question, I have no idea to be honest. The text looked like ChatGPT's for me (after many hours talking to it :D), that's why I asked to check what it thinks. But I won't be surprised if it's wrong and text is written by human. I wouldn't take it so serious, even if it's generated I don't see a problem here while it's correct (if so).

Reply on news.ycombinator.com