Hacker News Reader

Comment by ggm | Hacker News Reader

Yep. I've been accused of using ChatGPT to compose some of my posts here:

https://news.ycombinator.com/item?id=33930584#33930756

I did a quick detect on some your older posts: found this one from a year ago https://news.ycombinator.com/item?id=27568715 with 65 tokens and 99.14% fake. A quick googling shows sentences to be unique. GPT2 released on February 2019, so it would be nice to find older examples than that.

Of course, does the detector have problems when text from its training corpus is used? No idea.

Maybe your brain is just closer to an ideal GPT model than some others of us?

I assume the more clichés and boiler plate you write, that the GPT-like detection of fakes increases. However although I have played with the idea a little, I haven’t yet managed to get detected as very fake. Must try harder.

You can search by date ranges (though the date picker is kind of weird tbh):

https://hn.algolia.com/?dateEnd=1543795200&dateRange=custom&...

I think I've been subtly insulted again... fail the test, more closely aligned to the statistical model of an ideal GPT and using cliché.

I plead guilty to all of them. I clearly took one for the team, let the side down, disappointed at the finish, peaked early and missed by a country mile.

Try harder. Grow a backbone. (Hu)man up!

77 tokens, 88.25% fake. You are really good at this game!

And https://news.ycombinator.com/item?id=18576101 which you wrote 4 years ago (before GPT2) has: 79 tokens 99.86% Fake

Hopefully someone writes a script to run the detector over past users, because we are sure to find someone who naturally writes like GPT (after all, the training corpus came from somewhere!)

Also: no intent to insult - I would be even more amused to find myself detected as fake! I just love the topic - how much subliminal judgement to we make due to subtle word order choices? For example, how much more convincing could we be with some word or grammar changes?

After testing a heap of my own old comments using https://huggingface.co/openai-detector , I found one comment of mine https://news.ycombinator.com/item?id=17216586 with 65 tokens and 68.92% Fake. I don’t think I write much like other people. I also didn’t get any “bold” matches from https://news.ycombinator.com/item?id=33755016 (edited)

What was the hit rate on your own response above? The quine...

Hey, what tool are you using for detection? I'd like to run it on my own comments and on my blog posts.

Wait what if the reply that ggm is not GPT, is made by GPT?

Oh no pooh.. that's not honey it's.. recursion

Can I say that the response I wrote about 7% compounding is categorically not GPT and I fail to see what about it, perhaps should I say "them" since I wrote two distinct responses could be held to be artificial writing.

I truly expected it to be one of my more florid prose pieces. That was a minimally factual statement: 7% return on investment is astonishingly well documented, but under appreciated by people addicted to stonks, hodl and lambo talk. Oh dear. More cliché.. I'm doomed.

I think if we ran the GPT2 spotting model against anyone's comment history we'd find a couple comments that scored really high. I doubt it's actually related to your writing style.

Alternative hypothesis is that it's not about how you're writing but where. HN appears to have been used to train GPT3.5. I don't know if it was used to train GPT2, but it might have done. So your comments might be in the training set.

I already suspect my blog writings and standards work in IETF are, it wrote a pretty good CV covering letter/brief biography for me according to a relative (also a computer scientist) who tested it. He also noted it stuffed up his, claimed he'd written on work which was demonstrably untrue.

RFCs and associated mailing lists are pretty widely distributed and certainly RFCs aren't ring-fenced from GPT.

So if chatbot detectors are bad on people who are inputs to the ML can't that be added to the detector?

That's pretty cool. When I tried stuff like that, it was entirely based off of the prompt I gave it. I asked it to criticize my HN posts, and it came up with a story about how I was a charlatan; I asked it to summarize my views on Python exceptions in my HN posts, and it came up with a story about how I'm a world renowned Python expert. (I'm neither, but have written nothing noteworthy, so it's unsurprising.)

It could maybe be added to the detector but I think we're going to discover "did this output come from this model?" is ultimately undecidable. My intuition is that this is somehow equivalent to the halting problem, but that's a wild guess, and I'm not an AI researcher or computer scientist (just a humble SWE). Something like, if you start generating prompts that might take you to a part of latent space where this output could have been produced, you don't know when to stop generating longer and longer prompts.

I'd like us all to try and reserve the accusation that someone is a bot until we really need it. I know this is probably pointless, like asking a fish not to swim, but if we water down this language (which is already happening & perhaps inevitable, I kinda think "bot" just means "person who expresses an opinion I don't like, that I've heard before, and feel is repetitive" in some circles - eg in this video [1] they say "bots can be people") we're going to have a harder time expressing ourselves when we need it.

I'm a language descriptivist, so I'm bound to accept whatever the language does, but darn, I do find this particular evolution frustrating.

[1] https://youtu.be/LEvfdBF19kQ

I tend to agree. So.. thus was somebody else running a chatbot detector against my writings here. Ie it was an accusation leveled AT me, not made BY me.

If your complaint is that I committed a solecism referring to GPT trained models which can be found by hunting on Google for "chatbot gpt" I plead expediency and guilty.

I think we've misunderstood reach other, I wasn't criticizing you or anyone, I wanted to use your story to highlight a danger to everyone else that you've already experienced. I'm just shouting into the void that we shouldn't do something we're definitely gunnuh do, in the hopes that awareness will at least soften the blow.

The time is now, because bots are already among us. I don't see what good would come of accusing a bot though, that would be like arguing with a car when it's the driver that you should address in case of issues. We should document cases where bots are used to influence opinion though. Like these: https://www.propublica.org/article/how-china-built-a-twitter...

Also, and I'm only half-serious here, claiming the time to accuse bots hasn't come yet is just what a bot would say ;)

Just to be clear, I'm not saying the time hasn't come, bots have been among us as long as there's been a social internet, I'm saying we shouldn't turn it into a pejorative or throw the accusation around willy nilly, because that's spending powder we're going to need.

> Musk may well find himself restoring the various Ikea ball room for grown-ups in due course.

> But in a workplace climate with 20,000 former FAANG washing around like polystyrene debris in the canal estate.. maybe he has picked his time well.

I'd think "Ikea ball room for grown-ups" came from chatbot too, that's on you dude

Why?

Pretty sure that person was just being rude & didn't genuinely believe they were speaking to a bot, anyway. I think they probably meant, "you're so unoriginal, talking to you is like talking to a chatbot." Hard to know though, they didn't elaborate.

Someone specifically ran my text against a test program, it shows high positive for bot origin. There's a bunch of discussions about this, if you're in the training set, it may account for it. There's a case for the detectors flagging that. Scott Aarenson has said he's thinking about a background signal in buried semantic signals to detect.

The original one? Sure I think they just hate how i write and said so.

Admit it: if you could, you'd launch yourself into one. I know I would.

Reply on news.ycombinator.com