Misinformation has been a problem for all of human history, in its current form dating back to at least the Spanish civil war.
All the existing version of AI does is replace troll farms.
This isn’t a new threat, it’s a faster bullet.
Well, now we have $500 drones with an RPG strapped to it, and you CAN task a guided munition to each and every soldier and piece of equipment. There is no safe space in the field. You have to dig holes in the ground and hide from the sky. The lethality of war has gone way up.
These "AI" models are the same way. Before, it was expensive and effort consuming to run a scam or phishing or anything that requires social engineering. That's why the target the soft and valuable targets, like grandma, or people on MLM mailing lists, or large and juicy companies. But now, the cost of a phishing or scam campaign can be heavily reduced. Think of how often companies fail internal phishing tests, and those emails are usually pretty basic, low effort, trivially identifiable as phishing. It's a massive upgrade to troll farms. They don't have to limit their targeting as much, because cracking harder targets will be cheaper to attempt. It doesn't matter if it isn't actually more efficient or more effective, just the fact that targeting someone and initiating a campaign against them is cheaper is all that was needed to make this situation brutal.