The false positive rate of AI detectors and its effect on freelance writers
authory.com
authory.com
> They were terrified of Google downranking any AI-generated content, and because of the apparently clear results from their AI checker, they couldn’t be convinced otherwise.
It sounds to me like the client might even have believed the evidence that he provided them showing that he didn't use AI to write the work...
... but they're in the SEO game, and they were paranoid about a (so far hypothetical? I don't think Google have announced this yet) AI detection mechanism that Google might use to downrank them.
At which point whether or not he used AI isn't actually the issue - if his writing style is indistinguishable from AI, then in the client's mind the risk from another hypothetical AI detector within Google means that they shouldn't use the work.
I scan releases with a bunch of Anti-Virus software. Not to find any virus, of course, but if one of the scanning softwares "detects" a virus, some customer is also going to "detect" the virus, and we have a problem.
Windows has some good enough built in stuff, right? I can’t imagine using a third party scanner.
A lot of institutional "wisdom" in IT is that you need a virus scanner if you run Windows and because Defender ships with Windows it is therefore part of Windows and not a virus scanner. This isn't really based in anything tangible or empiric, just a learned knowledge that Windows needs anti-virus software and that there is good and bad anti-virus software and that's all you need to understand. Of course AV vendors are very guilty of feeding into this because their income depends on this lack of knowledge and being perceived as doing something vitally important.
Realistically the biggest attack vectors these days are (aside from server software vulnerabilities and browser vulnerabilities) someone launching a file they shouldn't have while ignoring every warning message along the way or entering their credentials into a phishing site or telling people things that they shouldn't know. None of these can really be helped with commercial AV software which is why a lot of AV software has moved on to MITM'ing SSL connections to intercept dodgy e-mails or bad links but creating their own potential security issues along the way.
The solution is code signing. Pay whatever you need to pay for a certificate and sign all your releases.
I’ma Googler, but had to try five search queries to find the name. Obviously didn’t try this in ages, but it used to be straightforward (and list which AV software was used in the scan).
What do they use?
But Google is probably a close second, with an even bigger archive of pre-LLM text (including the world's largest collection of digitized books), and supposedly a good LLM and lots of smart ML engineers.
But no matter what method google finds, whether it's some complex AI trained on massive amounts of training data or a simple n-gram analysis that ends up being telling, they aren't going to tell us. That would just accelerate the arms-race between LLM spam and LLM detection. If they have something that works (big if) the details will stay hidden, the best we are going to get is some kind of press release aimed at dissuading people from using AI on their websites.
And of course you don't tell people how your LLM detector works, don't provide a public API, and preferably leave it ambiguous whether it even exists. That solves most issues with LLMs using your detection method to get better in future versions.
That's counter to the business model of these LLM detection websites, but Google doesn't have a problem with keeping any potentially existing models inhouse and locked down.
If you detect LLMs 100% of the time correctly, it is irrelevant if you also incorrectly detect human written text as LLM text 80% of the time. "Detect them all and let god sort them out" will eventually fail as a business plan.
No, it means that someone was afraid they would use an AI detector similar to the one that was being used and downrank based on it; nothing in the article suggests Google is in fact using any AI-detector in its ranking algorithm, and I suspect that Google would be quite happy if people used AI (especially Google’s AI) to produce content that otherwise ranked highly on Google algorithms.
Google is probably just saying that scare off the SEO spammers from trying to use GPT4 to automate their jobs. I doubt they have a reliable detector (at least not one that detects based solely on the text content in the page).
Snake Oil Company: "This person's articles were probably -- to almost certainly -- AI-generated."
Crappy Client: "99% is very scientific and damning. Fire that plagiarizing writer."
Writer's Lawyer: "Sounds like libel with clear economic harm."
Writer: "Please, Snake Oil Company, you're ruining my career."
Snake Oil Company: "We know, and we don't care. Pay us some money, and who knows."
Writer's Lawyer: "Libel, with a side of protection racket."
Absolutely didn't happen. You might occasionally credit a story in the NYT or WSJ using an anonymous source—but when one appears in a blog by a "serial entrepreneur active in the content industry," flogging a subscription product aimed at exactly the sort of marginally-employed writer targeted by the story, I would expect the average HN user to be slightly more critical of the claim.
This might be a bias-confirming fake anecdote, for sure. But the effect is real.
My wife's fairly connected to the writing world and I've not heard of this being a thing at all. Doesn't mean it's not anywhere, but if it were widespread I think I'd have heard of it. Most companies are trying to push more use of AI tools in writing, as far as I can tell, and I don't even have insight into content-farm parts of the market, where I assume it's just all AI all the time now.
I've used LLMs for this as a sort of first draft. I edit the output but it's a perfectly serviceable way to get some words down and save me an hour or two.
If you needed to throw in a sentence or two ahout, e.g., what load balancing is, you're better off lifting a definition from an authoritative source than trying to come up with a "creative" rephrasing for its own sake. Standard terminology is standard because it works.
And the big middleman platforms siding with the client.
Some people prefer their anonymity, especially when they are put into the public sphere over disputes which can cost them future work. Notice the client wasn't named either.
I think this only exists to facilitate some already-bad heuristics, like the idea that nobody in-house needs to check the contractor’s work-product.
I tried it with my blog, and it was correctly identified by Copy Leaks as written by a human. I don't know how representative https://copyleaks.com/blog/ai-detector-continues-top-accurac... is, but certainly all of them are getting above phrenology detection rates.
Even with this guy, the problem wasn't that it misidentified him as AI. It was that "we can't tell" got turned into, "we can't risk Google thinking he might be AI".
Oh god, we've developed yet another shitty lie detector.
On the one hand... Companies spent the past couple of decades engaging in various SEO hacks to rank high on search results and OpenAI scraped the internet to train a language model. Theoretically, it seems possible that some of the SEO techniques at least partially colored the flavor of LLM-generated text, and an "AI detector" could pick that up. So if you do a great job writing SEO optimized text (wordy, structured, lots of repeated key words, etc.) you are more likely to be flagged.
But really.. "AI Detector" services are snake oil and will lead to the creation of "Anti AI Detector" services that offer protective spells against the original snake oil. See, we eliminate a bunch of jobs with AI but we create whole new disciplines of work that didn't exist before. "AI Generated Content Obfuscation Specialist - III - W2" coming to a job board near you soon.
That's why those types of services (including OpenAI's initial AI detector) often have huge legal disclaimers saying not to take it as 100% accurate.
https://arstechnica.com/tech-policy/2007/08/im-not-drunk-off...
I feel this is the only way this has a remote chance of actually working. However, I think you also need to make sure that the judge feels in on the joke as opposed to being the butt of the joke.
Courts (and authority in general) usually don't appreciate the "oh yeah, well from my point of view the jedi are evil" defense.
However, very occasionally, tricking the offense into skewering their own offensive strategy will work ... that is at least if the deciding authority is in a good mood and you can convince them you're the pragmatic choice.
As LLMs continue to improve, it would seem that it is only going to get more difficult to accurately distinguish between content which is AI generated vs human generated, so barring some kind of AI-detection breakthrough, these services will continue to plague students and writers with horrible false positive rates.
In 2016 a British company, Advanced Tactical Security And Communications (ATSC), started selling dowsing rods, basically made from old coat-hangers and an empty box as "long range explosive detectors".
From Wikipedia:
""" The device has been sold to 20 countries in the Middle East
and Asia, including Iraq and Afghanistan, for as much as US$60,000
each. The Iraqi government is said to have spent £52 million on
the devices """"
It's no surprise that the "AI detection" racket is going to be a field day for every hoaxer and huckster. But companies like Turnitin have long been selling "black-box" snake-oil "Plagiarism detectors" that have blighted higher education, ruined millions of student's lives, and created thousands of make-work hours for professors.
Really though, it's our fault that we keep treating technology as magic, failing to exercise radical scepticism and robustly challenge the low quality products pedalled by tech companies.
Professor: "The website said you used an AI." Student: "Well, here's a website claiming I didn't use an AI." Professor: "I don't know if I believe that website, let's look into the merits of how it works." Student: "Yes, let's do that." Professor: ...
I'm trying to think if you can indeed do something with a fair test.
I don't know, that might be incredibly difficult forever, or it might be something that someone has a breakthrough in and manages to do.
Perhaps it's only useful for particularly egregious examples, and wouldn't be able to detect an AI text that was subsequently edited. That would be fine. When I was a TA in college, we ran students' papers through a plagiarism detector, which would spit out some % number based on what it thought was the amount/likelihood of plagiarism. The number itself was bunk, but across a class of 30 people, if one or two papers had much higher numbers than the other 28, I could click and look at it's justification. Sometimes it was a meaningless false positive, other times it was "here are the paragraphs that were copied verbatim from Wikipedia".
One major issue is the feedback loop: There is lots of entrophy to hide a signal in regular English (whether you follow the methods of Stylometry or Steganography), and even an unmodified AI model has a stylometric signature. But if a significant portion of text we consume ends up being written by a handful of AI models, over time people will adopt many of these features. This is obvious right now, with ChatGPT having a certain writing style and some people unconsciously copying that style, and it will remain true no matter how subtle these things become.
well, why not train an AI on it?
I love this piece of history so much, thank you for sharing it! Adding this one to my bookmarks :-)
“The Waag is still open as a tourist attraction, and official certificates are available.”
Maybe with modern science we can make someone much lighter than they ought to be. (Put helium in their stomach?) Then they would be forced to issue a certificate that said that they could not guarantee that the person is not a witch :D
It's harder but probably not that hard.
If universities offer a learning environment that actually provides the students better learning than what they can get at home, the pieces of paper they hand out will continue to mean something. If they don't, they were already on a downward spiral kicked loose by online learning platforms handing out certificates.
The detector feels a lot like an excuse for whatever they're really concerned about.
The teachers that don't want to use them will continue to churn out students that score high and don't know how to articulate a damn thing. Which may have been the status quo anyway.
Edit: I didn't like my original post
Because of crippling anxiety.
So how do you account for that? Or autism? Or any other sort of neurological disability?
If I were a professor though, I would use a baseline for every student.
At the beginning of the semester, have an oral exam for a difficult topic that the students are expected to already know. Material from a prerequisite class would be good.
I would always give 100% for that oral exam if the student showed up. And that exam would become that student's baseline; their future oral exams would be judged based on that.
They'll probably be more nervous in front of the professor the first time than the last, so it would actually give them an advantage in later exams.
Probably still problems with that though.
But the other problem is harder to fix. I guess I could be much harsher in the first one, and go easier on the real exams later? Maybe that would reduce the number of students who do well on the first one?
Eh, I'm not a professor. Not my problem.
Btw, I was a professor and I saw students on both ends of the spectrum. Students who would bomb an oral evaluation because of anxiety but would do well in written exams. And student who would bomb written exams, but would totally ace an oral evaluation. Since, a professor's job is to create well-rounded individuals, courses should include a mix of different assessment types, so students slowly build up the skills to handle all of them. Of course, augmented by guidance and feedback on how they can build those skills.
We may see a rise of on-site homework.
The company is looking for text that ranks well on Google SEO. That is what the writer is asked to create. The company has cause to fear that anything that looks even remotely AI generated will be flagged and penalized by Google. Google does this regularly, is a black box, and regularly destroys companies when they do it it.
The company has found out that this guy's writing looks enough like AI that detectors aren't sure that he is human. They fear that where one detector is uncertain, Google may assume AI. Therefore the company has reason to believe that this guy is no longer able to do what they pay him for - producing text that will rank well on Google SEO.
Nothing that the freelancer is doing addresses that. The freelancer wants to prove that he's human. The company never doubted that. But their knowledge won't help them if Google's SEO rules change. His job is to provide text that will rank well on SEO. He isn't succeeding.
The freelancer tries to convince the AI detection tool that they are rating him badly. But the value that they are providing here really isn't, "Are you human." It is, "Is this content potentially going to be flagged by Google's search algorithm in the future?"
Yes, the next generation of LLMs will beat the tools. But that's just the next iteration of a cat and mouse game that's been going on ever since content farms figured out how to manipulate PageRank, and Google decided to penalize the behavior. And the rule of that game is that you focus on what you think will happen in THIS iteration.
Yeah, it really sucks for the freelancer. But the company is probably making the right business decision here.
It's not that next generation's AIs will beat today's detectors, but that there is no such thing as an AI detector. Just something pretending to be an AI detector that doesn't work.
Your proposal that the writer's 'writing looks enough like AI that detectors aren't sure that he is human' makes no sense. There is no 'looks like AI' that a detector can reliably distinguish from 'looks like a human'. All that's happening is the writer's work matches some completely bogus snake oil metrics made up by detector developers.
Given that, a bunch of well-SEOed content farms are bound to be killed for suspicion of being a ChatGPT written content farm.
You're often put in the position where your choices are to get fired or do what they want and watch their sales and search rankings tank. Unfortunately, 'I told you so' doesn't seem to cut much ice when that happens.
Does that warrant the author to be fired though?
Contrast this with the SEO focused writing that has been produced in heaps and heaps over the last decade. Good luck. It's actually hard to find any difference!
This could actually be considered good news. Maybe all of these "AI Detectors" are actually "bad writing detectors." If you're not producing anything original, in terms of real-world examples, style, or substance -- then maybe the writing ought to have been generated by AI anyways.
It's a game of cat and mouse in the sense that you can build LLMs specifically optimized for evading the current crop of detectors, but in my testing, they work pretty darn well in the general case. While they might not reliably pick on all LLM text, and while there's sometimes a couple of words in human-generated writing that causes them to output a low but non-zero probability of LLM content, they do not rate human-generated text as "99% AI". Especially not across multiple writing samples.
The most likely story here, I suspect, is that the person leaned on LLMs for commissioned writing and is now trying to save face. The secrecy of the models works both ways, right? And frankly - how often do you see people in HN, or people who do commissioned writing, admit in private that they're using ChatGPT? It's cropping up all over the place.
Note that I'm not commenting on the ethics, fairness, or transparency of tools like that. I'm just saying they work far better than you might be suspecting.
On the other hand, I tried three AI generated texts (>500 words each) and only one was marked as AI generated (it was so obviously AI generated that it would have stood out to me).
While it's flattering to be mistaken for an AI when you write at a high level, it's also a testament to your writing skills. However, it's essential to remember that human creativity, nuance, and emotional depth in writing still set us apart from AI. Even the most advanced AI models lack true understanding, personal experiences, and emotions, which are essential elements in many forms of human expression.
Embracing the evolving role of AI in writing and communication can be empowering, as it can assist in various tasks, from content generation to language translation. However, it's equally important to celebrate the unique qualities that make us human, such as our capacity for creativity, empathy, and storytelling. So, whether you're mistaken for an AI or not, your ability to write at a post-high school level is a remarkable skill worth acknowledging and refining.
Like what other defense is there? "No I didn't" prove it "I can't"
What are the chances that anyone who has a AI text detection tool ALSO has a AI generates text at human like speed with edits detector.
200 articles in 3 years of journalism seems very prolific. Can anyone speak to whats normal for a career journalist?
It's about four articles every three weeks. Doesn't seem extreme.
https://dublininquirer.com/2020/02/26/sam-how-many-articles-...
“Twenty years ago, in my first full-time job working for a daily newspaper, there was a period when I was turning out two or three articles a day – and I thought that was an unsustainably high number.
Today, young journalists have told me they’re asked to produce several times that.”
[0] Actual malice itself is a term of art. It does not require malice in the normal sense. Just a reckless disregard for the truth.
You don't run someone's writing through one of these detectors if you like them.
Are they actually good content or should he try harder to behave like a human?
This sounds like a defamation lawsuit.
This seems like an issue NFTs could take some pressure of off.
A new game: Sarcasm or Crypto Poe's law?
I legitimately don't know if I'm laughing with or at the above poster - all I know is I'm laughing.
The reputation would be an asset of yours, a post-monetary currency.
You know you're doing wrong, and doing harm—but you see a pitiful profit in it, so.
You want to go to sleep.
Google Bard to the rescue, Prompt: "Write a 200 word essay on the key ideas in Shakespeare's King Lear". The result isn't great. But it might pass an English teacher reading it quickly.
Can these be detected?
Bard: "Key Ideas in Shakespeare's King Lear
William Shakespeare's King Lear is a complex and powerful tragedy that explores a number of key ideas, including the nature of authority, the importance of family relationships, and the fragility of human sanity.
Authority and Order
King Lear is a play about the disintegration of authority and order. Lear's decision to divide his kingdom among his daughters based on their professions of love leads to chaos and bloodshed. Lear himself is stripped of his power and authority, and he descends into madness. The play also explores the corrupting influence of power, as seen in the characters of Goneril, Regan, and Edmund.
Family Relationships
King Lear is also a play about the importance of family relationships. Lear's rejection of his youngest daughter, Cordelia, who is the only one who truly loves him, is a central tragedy in the play. The play also explores the complex relationships between fathers and children, and between siblings.
Fragility of Human Sanity
King Lear is a play about the fragility of human sanity. Lear's descent into madness is one of the most moving and disturbing aspects of the play. The play also explores the relationship between madness and wisdom, as seen in the character of the Fool.
Other Key Ideas
Other key ideas in King Lear include:
Nihilism: The play presents a bleak vision of a world without meaning or justice. Self-knowledge: Lear's journey to self-knowledge is a central theme in the play. Old age: The play explores the challenges of old age and the loss of power and authority. Fooling and madness: The Fool and Edgar's disguise as Poor Tom provide comic relief in the play, but they also serve to highlight the madness and chaos of the world around them. Conclusion
King Lear is a complex and challenging play that explores a number of key ideas about the human condition. It is a tragedy that has resonated with audiences for centuries.
In addition to the key ideas discussed above, King Lear also explores a number of other themes, such as the nature of love, the importance of loyalty, and the power of forgiveness. The play is a rich and complex tapestry that continues to be studied and debated by scholars and audiences alike."
I don’t think current content generators will destroy many professions any time soon, but a lot of high-volume/low-value work is going to get consumed by them. This seems like an example of that work.
Hopefully, this guy didn’t expect to keep writing fluff into retirement and can find new footing with more sophisticated writing or pick up a new career with better opportunities.
It sounds so easy put like that. Hmm.. what do I have to do today? Get some mustard at the grocery store, pick up a new career, drop the dog off at the vet.
We can have sympathy for the guy and want to support him while also recognizing the practicality that he’s been walking down a dead end road and needs to change course.
Even without LLM’s, fluff writing is ripe for outsourcing to second-language writers in cheaper markets and gets more efficient as those markets optimize for it. It can be a job for a while, but it’s just not a viable career in the long run and opportunities were already narrowing.
Similarly, SEO often demands some graphic/any graphic to go with a story/blog/etc. and GenerativeAI is perfectly able to provide that--though lower-end royalty-free stock is pretty cheap in any case.
You assume that the detector actually works as advertised. Without seeing the authors content we have no way to judge in this case. However seeing that even openai struggles with this, i would question that this company has somehow found the holy grail.
In this case if you get wrongly classified you have no way to correct it. This thing might even make your writing worse because now you have to write it in a fashion that does not trigger the detector.
> I don’t think current content generators will destroy many professions any time soon, but a lot of high-volume/low-value work is going to get consumed by them.
In this case you have to compete against the generated AI content and fight against the black box AI detectors as well. You can improve the quality to fight the first. But how do you fight the second? Do i have to get a subscription to every single AI detector on the planet now?
but that's what new AI tools that can create good art/music/writing does. They boil down and automate the middle process of calculation.
what was previously "creative" is revealed as calculation from the God/Universe pov and as the range of calculable things grows, the island of im-a-real-human-bc-creativity gets smaller and smaller
defined this way... creativity ends up including some interesting domains like capital allocation and hedge fund management.
these allow for "expressions of freedom" unconstrained by calculation
> creativity ends up including some interesting domains like capital allocation and hedge fund management.
I hate this website sometimes.