Hacker News Reader

Comment by falcor84 | Hacker News Reader

I remember a professor of mine (I forgot in which class) saying that humans generally make up 3 kinds of explanations about a system's behavior:

1. When a system is very simple, we explain its actions in terms of its properties and external forces acting on it.

2. When it's a medium complexity system, we tend to explain its actions in terms of its design, putting ourselves in the designer's shoes.

3. When it's a complex enough goal-seeking system, we begin to empathise with the system itself, thinking why "it chose" a course of action.

I remember how impressed I was with this classification and how much sense it made in terms of how we're able to understand the world and predict what will happen next.

From this angle, much of modern computer software is clearly in category 3, and it just makes things easier for us to think of it as having a mind of its own.

View on HN

For category three I feel that goal-seeking (or the appearance of such) is sufficient for the anthropomorphism to kick in. The braitenberg vehicles thought experiment is a good example of this.

Quoting from wikipedia "For the simplest vehicles, the motion of the vehicle is directly controlled by some sensors (for example photo cells). Yet the resulting behaviour may appear complex or even intelligent."

https://en.m.wikipedia.org/wiki/Braitenberg_vehicle

But if you observe the behavior of a bacterium, operating under similar principles but seeking instead a salt or pH value, you would do well to describe it similarly. The bacterium is seeking a certain condition. It can not think, but it is a machine that acts with purpose and must be modeled accordingly.

I'm curious at what point an organism is considered to be capable of thought. From a mechanistic standpoint, it seems like the goal-seeking behavior of a simple organism could be modeled as a gradient ascent toward a local optima. But even complex organisms are still seeking certain conditions to maximize their ability to survive and reproduce. Is the existence of a brain enough for a creature to be considered capable of thought? How complex does an organism have to be before we no longer think of it as being a biological "machine"?

Or do we simply fall into the trap the GP described, where once something is complex enough that we don't understand what is going on from a purely mechanical perspective we consider it close enough to human to empathize with it?

A preread for your journey into the belly of the beast: https://en.wikipedia.org/wiki/Cognitive_ethology

What do you mean by thought?

A much simpler question can be answered: is it possible to be cruel to an animal? The accepted answer revolves around whether or not you can influence the behavior of the animal (or machine). Crabs naturally hide under rocks. If you shock them when they hide under the rocks they eventually stop hiding under the rocks. Meanwhile a bacteria, by itself, will never learn to associate a stimulus with a condition.

That may capture the capability for thought. There are higher levels past it, like learning to model the behavior of other organisms, and those models eventually turning into a theory of self.

I routinely anthropomorphize PID loops when tuning them.

One of the best nuclear astrophysicists I know thinks about stars as entities that want to stay alive -- "I'm running out of hydrogen, what can I burn next?" That approach yields the right phenomenology almost all the time.

That makes sense for really old things like stars, after all survivorship bias is a critical factor in most long lasting things we observe.

Well, yes and no. We say things like "the star wants to go nova" when we mean "stars at this stage in their development tend to go nova" - but the star doesn't want, any more than it dreams or fears to go nova. Anthropomorphizing might help us generalize about things made by people (who do want, dream and fear) though I don't think that's always the case, but I would say it isn't always useful for non-human (and especially non-living) systems.

Trying to follow that wiki article. Is the point that these vehicles appear to have goal-seeking behavior, but that is probably a projection on the part of the observer - and really its behavior is simply an emergent result of the incentives/effects programmed between sensor and effector?

Yes. IMO the thought experiment is really more about us than anything else.

For the discussion of your category 3 you might enjoy Dennett’s influential book, The Intentional Stance, which discusses how people reason from models like “the thermostat tries to keep the temperature between X and Y”).

https://en.wikipedia.org/wiki/Intentional_stance

Seems like https://en.wikipedia.org/wiki/Intentional_stance

I was just going to mention that. Another link:

https://plato.stanford.edu/entries/intentionality/

"However, in the absence of detailed knowledge of the physical laws that govern the behavior of a physical system, the intentional idiom is a useful stance for predicting a system’s behavior."

The human brain has certain data structures which it naturally parses and stores well. "Character with motivations taking actions" is one of them.

When you try to describe a complex or subtle thing concisely, you might find it hard. Even if the system is neither animal nor human, you too might notice yourself reaching for "character with motivations" to describe it.

Interestingly, extrapolated, that has predictive value for how robotic AI (i.e. androids) will be treated (eventually).

Reply on news.ycombinator.com