I had a post in a group get flagged by the algorithm for something "your politics shouldn't involve saying '[bad word] about [protected group]'".
I suspect the problem is there's nothing that would flag as wrong for a system that just defaults to such crudeness. So that's what happens.
But my boss apparently got complaints about bad matches, so he changed it to allow exact matches only.
The machine learning system ended up being a Rube Goldberg machine for linking people based on exact name match.
Some heuristic/standard algorithm works pretty well, but people see cases it didn't catch and think it could be better. An engineer/scientist takes a look and realizes the extra features required need a more complex/ml algorithm to add support for. Years later a model has made it to production, but there are now complaints about too lenient matching.
In my experience, ML works best in scenarios where there is either such immense data volume that .2% improvement is a real benefit (these are rare) or the very notion of a heuristic method simply wouldn’t work.
These contextual problems honestly show how shallow AI still is.
Banned for 24 hours, lol. Anyway. I think that is within bounds.
You just censored yourself replacing [bad word] and [protected group]. So surely it wasn't that harmless.