Hacker News Reader

Comment by wongarsu | Hacker News Reader

It's not even exclusive to LLMs. Giving humans seemingly innocent tasks that combine to a malicious whole, or telling humans that they work for a security organization while working for a crime organization, are hardly new concepts. The only really novel thing is that with humans you need a lot of them because a single human would piece together that the innocent tasks add up to a not-so-innocent whole. LLMs are essentially reset for each chat, making that a lot easier

We wanted machines that are more like humans, we shouldn't be surprised that they are now susceptible to a whole range of attacks that humans are susceptible to

View on HN

The assassination Kim Jong-nam is a particularly crazy example of this. Two women were put up to what they thought was, allegedly a harmless prank.

https://en.wikipedia.org/wiki/Assassination_of_Kim_Jong-nam

unless you know the target and trust the people asking you to do the 'prank' this is not a harmless 'prank'. if they thought they had rehearsed with the target then i think they have a strong defence but i think they were extremely lucky to have avoided a murder conviction. what they were doing is assault even if it was not poison unless they had the consent of the target.

Breaking tasks into innocent subtasks is a known flaw in human organization.

I'm reminded of Caleb sharing his early career experience as an intern at a Department of Defense contractor, where he built a Wi-Fi geolocation application. Initially, he focused on the technical aspects and the excitement of developing a novel tool without considering its potential misuse. The software utilized algorithms to locate Wi-Fi signals based on signal strength and the phone's location, ultimately optimizing performance through machine learning, but Thompson repeatedly emphasizes that the software was intended for lethal purposes.

Eventually, he realizes that the technology could aid in locating and targeting individuals, leading to calls for reflection on ethical practices within tech development.

https://www.rubyevents.org/talks/finding-responsibility

The book Modernity and the Holocaust is a very approachable book summarizing how the action of the holocaust was organized under similar assumptions and makes the argument that we’ve since organized most of our society around this principle because it’s efficient. We’re not committing the holocaust atm as far as I know but how difficult would it be for a malicious group of executives of a large company quietly directing a branch of 1000’s who sleepwalk through work everyday to do something egregious?

> Giving humans seemingly innocent tasks that combine to a malicious whole

Isn't this the plot of the The Cube!?

Eagle Eye too, with Shia LaBeouf, although people in that story are constrained into doing specific small tasks, not knowing for whom, why or what is the endgame.

I actually like that plot device.

I wouldn't call it the plot of the Cube, more like the setting/world-building.

The plot is about the people trapped in the Cube trying to figure out their situation and get out.

The construction of the Cube is kind of a backstory, not the main part.

Reply on news.ycombinator.com