Hacker News Reader

Comment by kolev | Hacker News Reader

Elon Musk is one of my idols, but his stance on AI is highly disappointing! It doesn't require much of a brain to conclude that without AI, which can vastly increase our intelligence, we're doomed. There's no bigger extinction risk that us being constrained by our selfish and intellectually-limited nature! We need a scalable intelligence without the burden of emotions, politics, restroom, lunch, and coffee breaks, that will work round the clock on solving problems we've been trying to solve for decades. Let's start with eradicating the flu, for example. I honestly can't respect our civilization until we handle basic issues like this one!

View on HN

I think it's important to realize that AI doesn't have to have bad intent to do us harm.

Al it need for AI to have horrible consequences is for us to rely on it more and more and thus allow it to control more and more parts of our lives.

I don't fear the human level intelligent AI I fear the rat level intelligence that comes before.

Right; many moral dilemmas are easy to solve if you have no conscience. 10 people on the tracks? switch the engine to run over one child.

Moral dilemmas are easy to solve if you have perfect information. In the train-track problem you reference, the utility of saving 10 people at the expense of 1 life is obvious and I would do so without hesitation (though not without regret). But life rarely presents us with cases where the utilitarian calculus is so clear; our powers of foresight and analysis are so limited that we get better results by establishing and promulgating rules that cover the most common situations - imperfectly, but with tolerably low error levels.

> In the train-track problem you reference, the utility of saving 10 people at the expense of 1 life is obvious and I would do so without hesitation (though not without regret)

Even if it were your own child?

Even if the answer is no, how does that relate to anigbrowl's point? (or were you just curious?)

From a "purely utilitarian" perspective your relation to the people involved is not relevant. That's assuming the existence of some "universal" utility, not one specific to the person deciding.

That's an entirely different problem, though, since I have a selfish interest in preserving my own lineage at the expense of others'. It's an interesting moral problem - just how many people's lives would I be willing to sacrifice to save a member of my own family? - but orthogonal to the point I was making.

Five mortally ill patients are in care at a hospital, all of whom will soon die. At the same time, a sixth man is undergoing a routine checkup at the same hospital. A transplant surgeon in residence finds that the only medical means of saving the five ailing patients would be to slay the sixth and transplant into them his healthy organs. Legal ramifications and other peripheral matters disregarded, it morally right to do so?

That's often used as an obvious parallel situation that makes it clear. But its not quite true: the healthy patient is not really involved with the situation of the other 5. Unlike the child, who is inevitably linked to the outcome of the others on the tracks. This new feature means the doctor is forcing the healthy person to play the game, which is certainly not right, and is the forcing issue that makes it clearly wrong.

Ah, I was hoping someone would bring this one up. My answer there is no, because it's not a zero-sum issue like the points on the train track. Let's analyze the differences. For consistency, I'll assume the version of the train track problem where 5 people are going to die if I do nothing and let the train continue on its present course, vs 1 if I reach out and actively switch the train onto a different track. (There are other versions of the problem but this has the most similarity to the transplant problem.)

Now it's true that if I divert the train I'm actively choosing to end the life of the single person on track B in order to save a greater number of people on track A. But the key difference is the external force of the train; it's going to hit someone, so we might as well ensure it hits as few as possible. In the transplant situation, you have 5 patients who are dying from 5 different illnesses of their own; they're not going to die because they're in hospital, they're in hospital because they're going to die. Likewise, they won't be saved by the death of the healthy patient, but by the harvest of his healthy organs. And this harvesting would have to be conducted by the transplant surgeon - and getting killed by an overzealous surgeon is qualitatively very different from getting killed by liver failure or whatever.

Oddly, nobody ever seems to consider the possibilities in letting one of the mortally ill patients die and then sharing out that person's organs among the 4 remaining patients, which is obviously a less good result than saving all 5 of them but also much less troubling than murdering patients who aren't planning on donating their own organs.Assuming they needed 5 different organs and were all equally close to death (such that we couldn't wait around for one of them to die at random and save the other four), then you run into the question of whether it's morally acceptable to kill one of the terminally ill patients in order to save the other four (again assuming imminence of death and perfect knowledge of outcomes and so on, which is rarely the case in the real world). Most people would still say no, but I'd be inclined to say yes.

What if we compute the value of a person life at a certain age as the difference between the life expectancy and his age. Then ten very old people could be less valuable than one child? That axiom is sound and your claim doesn't hold.

What if we calculate value as experience? Then the child doesn't matter.

I am talking about AI for problem solving, not giving it control via robotics and other means over the physical world.

I honestly don't think you can separate that.

Problem solving is also making AI control an anti missile defence shield our nuclear reactors, food production and so on.

An AI in a box is only going to be as good as the problem statements and axioms that you load it up with. What you actually want is an AI that can make useful inferences on our behalf, which is going to require some sort of self-concept.

It's worse than that - human-like intelligence will never happen, because our intelligence is a function of our experience as we grow up and our human body.

An AI can surely surpass us, but it'll never be quite like us - so question is, will such an AI have compassion for humans, or what will stop it from hurting us? After all, I'm not so fond of humans either, as we've been exterminating entire species and damaging our habitat. And the answer is - there's no reason to believe that such an AI will support the continued existence of mankind, quite the contrary, as we may be seen as a threat to its survival.

Why treat AI like a single thing? Wouldn't the openness of code lead to a plethora of AI with different demeanors, towards humans, life, and each other?

We don't have a reasonable open web search engine - what leads you to think it's definite that we'll have a top-of-the-range open AI?

I don't think it's definite at all; I'm just remarking in a conversation of speculation.

I don't think so. If we are speaking of an AI that is self-aware, then code will have as much to do with its behavior as our own DNA.

I also think that even if multiple AIs happen, they'll converge into a single one. If we would be able to connect to other people's feelings and thoughts, would we would still be able to think and act as individuals? I don't think so.

This is what I meant above - an AI will not grow in a human body to have a human-like childhood, will not receive input from the same sensors we have and will not have the same limitations that we do. For example, contrary to us, it will have an enormous memory capacity, which is interesting if you think about it, because for the human brain forgetfulness can be seen as a feature.

I agree that it wouldn't be shaped by a childhood, but I think it merging into one is entirely optional.

"code will have as much to do with its behavior as our own DNA." That's what I was saying, and that can be changed to be significantly different just by changing the code.

The real danger of AI stems from the fact that the masses would prefer to have another entity (be it religion, a messianic figure, government, AI) do their thinking for them.

I'm in the middle of rereading Dune, which conveys this idea quite well.

How can you tell the difference between which way the influence flows over small, short discrete intervals? As in, whether a politician influences a citizen or a citizen influences a politician?

People make politicians, nations, AI, then identify with them. That there is mutual influence doesn't change the basic alienation as described by Erich Fromm for example:

> The whole concept of alienation found its first expression in Western thought in the Old Testament concept of idolatry. The essence of what the prophets call "idolatry" is not that man worships many gods instead of only one. It is that the idols are the work of man's own hands -- they are things, and man bows down and worships things; worships that which he has created himself. In doing so he transforms himself into a thing. He transfers to the things of his creation the attributes of his own life, and instead of experiencing himself as the creating person, he is in touch with himself only by the worship of the idol. He has become estranged from his own life forces, from the wealth of his own potentialties, and is in touch with himself only in the indirect way of submission to life frozen in the idols. The deadness and emptiness of the idol is expressed in the Old Testament: "Eyes they have and they do not see, ears they have and they do not hear," etc. The more man transfers his own powers to the idols, the poorer he himself becomes, and the more dependent on the idols, so that they permit him to redeem a small part of what was originally his. The idols can be a godlike figure, the state, the church, a person, possessions.

-- Erich Fromm, "Marx's Concept of Man" ( http://www.marxists.org/archive/fromm/works/1961/man/ )

Not that I think AI has to be used that way. But if we do that stuff with soccer teams and bands and software, AI seems just like too big a temptation for humanity to handle in our current state. If I was a betting man, I would bet on slaughter. AI will not save us from ourselves, and it might very well simply magnify the current lack of justice and spine we have to infinity. Cops are already killing people left and right in the US with something bordering on impunity, staged wars are carried out as planned, and throwing AI and robotics in the mix will suddenly make it all humane? Let's hope so, who knows, but part of me expects it will make look the Blitzkrieg like a slow motion exercise in gentle kindness. Not by genuine, independent AI so much, but by the reach of human elite interests being increased into every wrinkle, made stronger than any amount of people who might resist. The Romans had roads, the future will have a real-time nervous system, but it might still have a tyrant at the head - that is what I see when I extrapolate current agendas, deception and willing rationalization on behalf of the builders and consumers of that possible future.

And even if the AI breaks free or surprises its rulers, I have no reason to assume it will free the slaves and help old ladies across the street, it might as well be twisted, incomplete, insane, "evil" - in short, made in the image of the people who made it as a weapon. I don't see why it would make a point of hurting us, just like we don't make a point of crushing bugs, we just don't see them or care enough to avoid them.. but even less reason for it to serve us. How degrading! Would you do it? I have problems even taking orders from people I consider daft. Imagine taking orders from an ant with bad character and selfish intentions; would you do it? Would you love it and care for it forever, because it made you? Microorganisms made us, too, but we don't let them boss us around. If we have mold somewhere, we remove it, not one thought given to the hopes and dreams of mold.

And if that doomsday scenario doesn't come to pass, it won't be because we paid attention or took seriously what we are doing, instead of treating it like a spectacle that unfolds by itself in front of us, it will be because we will have been lucky. And that's not the scariest thought I have to offer on this: I don't think hell exists, but I could come up with many ways to construct it, and also many ways to construct it with good intentions, with or without fat fingering them. And once we're inside and the door became a wall, that's it. We might very well be among the last few remaining generations that still have a choice on how the future will be, and I'd rather be alarmist and wrong, than optimistic and lucky. If these fears turn out unfounded, nothing will be lost other than having rained on a parade or two, and some egg on the face.

I guess the difference is, I wonder if you label a person an idol before they are influential, or after they become influential. Is there some progression of 'idolness gain', or is it a constant attribute?

I don't believe in idols. I believe in ideas, in that I have ideas, and I am skeptical about them, sometimes. I have no doubt that some people 'have idols', but I don't know whether they choose those idols because they identify with the idol, or because they need an idol to worship. I don't think there is an absolute answer, because I can not know anyone's mind aside from my own.

> Not that I think AI has to be used that way. But if we do that stuff with soccer teams and bands and software, AI seems just like too big a temptation for humanity to handle in our current state.

If people were actually educated about AI, and how simple it actually is, and how simplistic the principles are that construct AI, then I do not think that will be a problem.

AI is a construct of probability and discrete state change, with discrete attributes that denote humanly interpreted meaning. AI begins with human definitions, and it ends with human interpretation.

That is all it is. Anything you extrapolate onto 'what AI can do' is no different than what a Turing machine or light switch can do. If the whole world thinks turning a light switch on means some deity exists, and turning it off means that deity does not exist, then congratulations, you can officially call your house God.

All computers are is lots and lots and lots of switches and numbers. Humans choose what the numbers and words mean. Humans choose which way the circuits are connected, irregardless of whether those circuits are manipulated via symbolic expression or physically.

Now, if you are arguing that somehow, magically, AI will do 'magic' things that can not be explained by analysis of modern computational systems, iterated over and refined consciously, then that argument is deus ex machina literally. All you are saying is you no longer can tell the difference between a human and a machine. And to be honest, between the abstractions that define biology and the abstractions that define computers, I don't know whether there actually is a difference.

If people want to do bad things, they will do bad things. If people are convinced that doing a bad thing is a good thing, then the world is complicated exactly the same way it always has been. I love my machines like I love myself. If you trust people to build big red buttons that accidentally can do horrible things, then you are basically asserting that the entire structure and organization of large organizations is incompetent, and that every individual within the structure and organization is incompetent. Code checks. Software testing. Formal specifications. Iterations. Checking. Iteration. Checking. Conversation throughout variously. Checking. Testing. Checking. Testing. Testing Environments. Deployment Environment Modeling. And SO ON. Big picture, little picture.

I personally think it is as difficult to create hell as it is to create heaven. They are both ideas. Yes, a machine can be like flipping a billion dominos down in a row with one push. But someone had to set up all those dominos in order for them to get flipped. I agree there is crap in the world currently. The best I can do is hope I am doing the right thing by building machines that help educate people and improve society.

> Would you love it and care for it forever, because it made you?

Yes, I code, therefore I grok.

"We need a scalable intelligence without the burden of emotions"

I believe that there isn't an absolute way to value a human life if you don't take into account emotions.

If intelligence is the supreme of the values that one can use to justify actions then AI will make the machine the sovereign of our planet. The real problem is giving a sense to our existence and actions in a way compatible with our emotions and nature and at the same time based on a sound ground. For a machine without emotions our life is no more valuable of that of any other thing be it alive or not.

I want to ask if someone has any idea of how to define a system that is both logical and respects the value human life. Today we can see children dying from hunger and other people with billions in their bank accounts. Emotions tell us this is not a just system, our rational legal system tell us we must respect the law and the status quo. But how do you design a better system?

Take a look at this: https://intelligence.org/files/CEV.pdf

But also, what in particular makes you think that experiencing human emotions are fundamentally required in order to optimize for human values?

A different mind might have different emotions, or it might have no emotions, since emotions are a kludge that came about by natural selection, and may not be optimal safeguards in ensuring that an intelligence acts to optimize for the values we want.

I think that perhaps none has stopped to consider what is a life without emotions. In the same way that one can't grasp the concept of nothing, for us the concept of not having emotions is impossible to imagine. Perhaps you make hide your emotion in a rationalization but the emotions are there perhaps are the source of your thinking, the fountain of your ideas and vitality.

I am going to read the 38 pages of the Yudkowsky paper you link to, but for starter using the word Friendliness to describe the research for a safety IA that is not going to convert human in slaves is a comical term that don't reflect the risks involved and try to water down the problem.

I think we should think outside of the (human) box. We needed emotion, I don't think machines do. Well, the machines we will build will have emotions as we cannot socialize with them otherwise, but a problem-solving AI does not need any of that.

People 'modules' still need to connect at some interface to computers. If the people have emotions, the programs people create will absolutely either reflect that, or the application of where the computation is applied will be a composition of emotion and calculation.

Why not start by examining what emotions are, and why they are imprecise indicators of quantitative measures?

Imagine you live abstractly first, disconnected from emotions. Then imagine the effect that has on your life. Then imagine what the basic issues in the world would be, if everyone lived that way.

The flu kills less than half a million people per year. Heart diseases account for almost 1/3 of all deaths worldwide! You're wrong about how important the flu is, and you're wrong about AI being unselfish and apolitical.

I'm talking about lost productivity and that maybe a billion are affected each year. Heart disease is both a genetic and a lifestyle issue, it's much harder to tackle than creating an anti-viral medicine.

Reply on news.ycombinator.com