Hacker News Reader

Comment by ajobaccount2017 | Hacker News Reader

ajobaccount2017

8.2

A Master Programmer passed a novice programmer one day.

The Master noted the novice's preoccupation with a hand-held computer game.

"Excuse me," he said, "may I examine it?"

The novice bolted to attention and handed the device to the Master. "I see that the device claims to have three levels of play: Easy, Medium, and Hard," said the Master. "Yet every such device has another level of play, where the device seeks not to conquer the human, nor to be conquered by the human."

"Pray, Great Master," implored the novice, "how does one find this mysterious setting?"

The Master dropped the device to the ground and crushed it with his heel. Suddenly the novice was enlightened.

If you want to compete with AI, don't make humans easier to hack.

View on HN

Humans are typically the weak link in a security strategy... so I think this would make them harder to hack.

We're easier to socially engineer, but so far humans are monstrously hard to hack; we're as likely to break unpredictably or not at all, as be broken.

I would argue that humans are quite easy to hack although perhaps not in the way you were thinking. Our perceptions are quite easily and reliably hacked even when we know it is happening, as demonstrated by the success of stage magic and optical illusions[1]. At one point, there was a great deal of fear that companies and adversaries could influence large groups of people with subliminal messages[2]. Although the efficacy of these techniques are somewhat in doubt, on a more mundane level, most people are used to attempts by advertisers and marketers to subvert our desires and preferences to buy certain products. A great deal of research and money has been spent on essentially hacking our desires and exploiting our brain's response to intermittent reward and social cues. This has brought us product placement in movies, celebrity endorsements and video games that produce changes in the brain not much different from those found in drug addicts. Scientists have engineered fast food to exploit our evolutionary desire for sugars and fats -- something that was good at one point but now serves only to make us and fast food executives' wallets fat. I would consider all of these a type of human hacking simply due to the reliability of their effect, if not on an individual level, certainly on groups of people.

[1] - https://en.wikipedia.org/wiki/Optical_illusion [2] - https://en.wikipedia.org/wiki/Subliminal_stimuli

Optical illusions are neat, but I don't think it rises to the level of a "hack" in the sense that anyone is talking about here. Subliminal stimuli is, as the massive warnings on the article suggest, utterly unproven. There is a difference between being swayed or marketed to, or just too lazy to disengage from the onslaught of marketing... and "hacking". If hacking humans worked, the marketing wouldn't be necessary in the first place.

Although some of my analogies were a stretch, I think it might be valuable to regard these types of manipulation as hacking. If nothing else, to make us aware of our own susceptibilty to con artists, fake news and government influence. The reason these techniques are effective are very similar to the reasons hacking methods of computers are effective; they take advantage of systems that evolved or were built for one purpose in order to use them for another, often to the detriment of the victim.

For example, the existence of optical illusions and stage magic are a necessary consequence of particular limitations of our visual system and attention. One could predict the existence of new, never before seen optical illusions purely from knowledge of the way the brain processes visual stimuli. For more information on this, see the works of Roger Shepard[1][2] who did a lot of research into the psychology of perception and mental representations.

This has ramifications for not just human psychology but artificial intelligence. If we want to build a computer system or robot that can process visual information quickly like humans and animals do, then we may very well have to program them with the same simplifying assumptions that humans and animals use to make rapid visual processing tractable[3][4]. A consequence of this may be that these computer systems will be subject to the same optical illusions as humans as a necessary consequence of limited attention and computational resources. Furthermore, the misperceptions that make stage magic possible may be possible to induce in any system that can only pay attention to a subset of the visual stimuli given and that must make assumptions about the intentions of the subject being viewed. These assumptions and inferences are usually accurate under ordinary circumstances when the subject is not trying to deceive the observer, but a clever subject could engineer circumstances where the observer has no choice but to be either deceived or accept that their perceptions have no logical explanation -- hence the woman appears to be sawed in half even though we know this is unlikely; there is no visual information to disprove that she was (the lack of blood is evidence that she wasn't sawed in half but this is only evidence from our past experience with people being cut by blades).

We are susceptible to influence by fake news and celebrity product endorsement due to our evolved preference for information coming from "authority figures" and sources that agree with our existing views[5]. Now, normally one may hesitate to call exploiting these systems "hacking" because the exploiter often doesn't know that that is what they are doing just as someone may stumble upon a new computer exploit without knowing exactly why it works.

Again, I would argue that it may be to our advantage to think of the targeted exploitation of these innate tendencies as a type of "hacking" if only to make it more likely that we can avoid being influenced to beliefs and behaviors that may not be in our long term best interest.

[1] - http://im-possible.info/english/art/classic/shepard.html

[2] - http://rumelhartprize.org/?page_id=110

[3] - http://ilab.usc.edu/publications/doc/Miau_etal01spie.pdf

[4] - http://www.sciencedirect.com/science/article/pii/S0004370202...

[5] - https://en.wikipedia.org/wiki/Authority_bias - see also https://en.wikipedia.org/wiki/List_of_cognitive_biases

Reply on news.ycombinator.com