Given the overwhelming amounts of slop that have been plaguing search results, it’s about damn time. It’s bad enough that I don’t even down rank all of them, just the worst ones that are most prevalent in the search results and skip over the rest.
Since even classical machine learning uses BERT based embeddings on the backend this problem is likely wider scale than it seems if a search engine isn't proactively filtering it out
Is this a term of art? (How is perplexity different from complexity, colloquially, or entropy, particularly?)
A naive way of scoring how AI laden text is would be to run n-1 layers of a model and compare the text to the probability space of tokens from the model.
It works somewhat to detect obvious text but is not strong enough a method by itself.