Crazy that Twitter still sells ads and soooooo many of the accounts are bots. They have an obvious interest in inflating the total numbers.
It's a super, super simple problem. If they really cared, they'd eliminate bots entirely. Not that hard.
It's a super, super simple problem. If they really cared, they'd eliminate bots entirely. Not that hard.
No offense, but that's incredibly easy to say when you're not the one responsible for doing it.
The researchers who keep publishing these papers and the patsies in the media who keep repeating them are not identifying bots, in fact they can't know what their own accuracy is because they don't have any way to force verification of accounts. But it's apparent from reading their work that they're usually just identifying humans.
I used to do anti-bot detection and anti-spam work for Google, and have written a takedown of one of these media articles about Twitter bots:
https://blog.plan99.net/did-russian-bots-impact-brexit-ad66f...
The paper I studied was riddled with nonsense, fraudulent claims, claims that didn't match their own data, abuse of logic and statistics and massive political bias. It is typical for this space.
This is especially true because real bots (vs the bots that exist in the imagination of political journalists and academics) usually want to sell things. They're quite boring. They aren't engaged in "propaganda". The belief that spammers are spending lots of time and money trying to flood Twitter with political opinions isn't grounded in any sort of reality, and frequently leads to embarrassing claims, like this guy who was pegged as a "Russian bot" but who turned out to be a carpark attendant in Glasgow:
https://order-order.com/2017/11/15/byline-outs-russian-troll...
These claims about spambots should be seen in the context of a US political campaign in which the losers seemed to be struggling to understand why their preferred candidate lost. The idea that the population has been bulk brainwashed through Twitter doesn't have any basis in psychology or computer science, but is an attractive way to avoid engaging with policy issues.
CAPTCHAs are at best a game of cat-and-mouse, and are always open to relay attacks anyway.
You can still verify in-person, and even check documents, but no web company would even consider doing that, even for something like an optional 'fully verified account' tag.
For small websites, even a completely trivial email check is enough to keep out most spammers.
Eliminating the bad ones with few false positives is hard.
Technically speaking, I'm not sure it's even possible to tell for sure which accounts are bots and which aren't. How can they tell?
My day job is information security, specifically working with a SIEM to correlate many diverse logs from many diverse systems and figure out what really happened using many pieces of individually benign data. None of these things are themselves indicators of bots, but the more you start to trip these rules, the more bot-like your behavior becomes. Eventually it paints a picture that shows no human could reasonably be behind an account that routinely posts two or more tweets at the same time, never engages in follow-ups, is only followed/liked by other suspicious accounts, and has a user agent of Python 3.7 coming from a source IP on aws.amazon.ru. You show them a captcha and if they fail or bail, you've got 'em.