Comment by perihelions | Hacker News Reader

Comment by perihelions | Hacker News Reader

The most obvious way the US national security industry could use LLM's right now is simply to spam foreign adversaries with chatbots. That's their greatest strength right now—a use-case they have amply proven themselves for.

This paper comes off as eager to avoid this topic: they (briefly) talk about detecting foreign LLM spam, which is called propaganda, but sidestep the idea of our own side using it. If we were considering talking about that, we wouldn't choose negative-sentiment descriptors like (quoting the paper) "nation-state sponsored propaganda", or "disinformation campaigns"; we'd use our own netural-sentiment jargon, which is "psychological operations" ("psyops") [0].

That we're not widely debating this question right now *probably* means it's far too late to have a chance of stopping it.

edit: Or, to rephrase this as a question: Is it ethical to spam another democracy with millions of chatbots pretending to be citizens of that country—if the effect is to manipulate those citizens to not go to war with our own side, saving our own lives? Is that an atrocity or is that legitimate warfare?

[0] https://en.wikipedia.org/wiki/Psychological_operations_(Unit...

I'm pretty sure that before the era of LLM, hired personnel and computer chatbots already exists on major social media platform to serve as "opinion influencers", either for political or commercial purposes. Just that chatbot algorithms are kid of naive, and have been "fished" on several occasions....

The usage of LLM is too obvious, and way cheaper than hired personnel, so it's like "prism". They don't want us to talk about it because it's been going on for so long...

They have, and it scales orders of magnitude better with LLMs.

The platform you are referring to is called "Reddit", one of YC's portfolio companies.

Or, to rephrase this as a question: Is it ethical to spam another democracy with millions of chatbots pretending to be citizens of that country—if the effect is to manipulate those citizens to [take any action advantageous to US national interest]...?

Just, Devil's Advocate, but ethical or not, that's what we should be doing and what we are doing. Every nation has its sock puppets out there, our job is to stop everyone else' sock puppets, and do everything we can to extend the reach of our own sock puppets.

That's not inherently true. If there were a way to reliably destroy all the sock puppets, we should, and the world would be better off. For instance, reliable bot detection, or mechanisms by which major social networks could detect and prohibit bot-like activity that isn't labeled as such.

charge one penny for every post. most people can afford it. bots become less cost effective and you'd be able to trace the source of funds.

Even if they could afford it, they won't. The UX friction of charging money would send engagement off a cliff, even if the nominal charge was $0.

that's why it should be mandated by law for national security reasons!

Only half joking, I feel like the majority of comments on the internet are garbage and not worth the time to read, and increasing friction with a nominal charge isn't necessarily a terrible thing.

It's a good thing for most people, just not the "platforms" trying to show growth.

This is a nice idea, but we already know that organized criminals and state actors have no problem spending money to make money (or influence public opinion).

On top of that, social media isn't like email where a potential transaction fee increases the cost per person reached - a single post on a hot topic could get millions of views.

I don't think you understand the Devil's Advocate rhetorical device. It's not just a thing you say before you state a morally reprehensible opinion you hold.

Imo there's a difference between propaganda promoting functioning democracy and propaganda promoting conservative-at-best authoritarianism.

Why stop there? Send missiles! Heck, send bioweapons and dirty bombs. It might not be ethical but if others doing it we might as well join 'em.

(This is sarcasm. I don't want to live in this world.)

We do.

I'm sure they're already used against both foreign and domestic populations.

Oh, national security professionals aren't going to be talking about psyops, offensive applications and etc, because such things make a given state look bad - they're an offense against democracy and respect for facts and they make the overt media of a given nation look bad. But hey, leave it to HN commentators to root for taking the gloves off. Not to worry post, I'd bet dollars to donuts the actual classified discussions of such things aren't worried about such niceties. But even more, in those activities of the US and other secret states, that have come to light, these state have propagandized not only enemy populations but also their own. After, have to counter enemies trying to nefariously prevent wars as well.

Reply on news.ycombinator.com