I'm not naive enough to think that these companies don't already have some internal detection tools, but they clearly are not being used enough. Lately researching anything using agents has been a pain (especially if the query leads up to buying intent) - the sources quoted are often AI slop, which are then used to deterministically argue a point. On the first challenge to logic, the reply is usually an apology followed by a tangential search and agreement on the user argued point - sometimes even a hedge. There is a simple problem embedded at the center of this, that at least partially explains this degradation - AI generated text. There is now arguably more AI generated text on the web than human writings, and platforms that used to be very credible (attracting high quality technical writers) now seem to be full of AI generated slop (someone has to call them out). It's a pattern I'm sure most people are noticing under the hood - query -> beautiful prose -> suspect sources. You challenge the answer, and all of a sudden it's a wildly different conclusion. To me that just sounds unreliable, because you could potentially argue with it to generate any conclusion you want - doesn't sound like a reliable brainstorming partner anymore. Not to mention it's preference for believing structured AI prose output as high quality writing. Perhaps it is simply a byproduct of the baked in agreeability or "dooms-day" protections baked in to system prompts. This makes it even more important to have sources quoted by bots verified for authority and origin. The authority test is being easily faked by rampant SEO abuse on once reliable blogging platforms (I won't name them but you know who) - apparently one bot - sorry user - singing product praises is enough for chat to recommend that product, mind-boggling. And one biased - sorry tailored - article comparing a product to 5 competitors is enough to determine which is best. If this is not a crisis, idk what is.
The detection problem: Using conventional techniques like duplicate detection, repetition and similarity filters on AI generated content is a lost cause - there's enough probabilistic variance in them to generate millions of different copies.
One solution to the problem: Use an AI detector to adjust result ranks. By adjusting the ranking using AI generated labels, not only will engines prioritise genuine human writings - they also potentially refer to a wider range of opinions on a subject, improving diversity consulted in answers. I know it sounds like an over-simplification, and it probably is - but to me it does sound like a first step in the right direction.
Taken from https://t9xt.com/writings/why-google-microsoft-anthropic-and-openai-need-to-build-their-own-ai-detection-tools-before-it-i