AI Evangelists Scare Me
gavinhoward.com
gavinhoward.com
This has already fully happened under the name of 'algorithms.' Not necessarily like classical computer science algorithms but like 'the algorithm rejected your loan' or 'the algorithm said your insurance rate is $517 per month' or 'the algorithm declines your registration application'.
And AI will only accelerate the trend.
So for instance I'm using an AI to evaluate college application essays, trained on a big database of them someone is selling.
The grading AI will grade essays similar to the best rated training essays high, and the essay generating AI that naughty applicant is using will generate an essay as similar as possible to the training data in the ways an AI cares about (word frequencies?).
However a human applicant writing an equally good essay will get a lower grade from the AI because it is looking for similarity not "goodness" exactly.
This creates a situation where you need to pay for the most popular AI in order to win contracts, get jobs, planning permission, grants... it's a network effect like social media.