Hacker News Reader

Comment by galaxyLogic | Hacker News Reader

> It doesn't have a goal.

Right that is what AI-phobics don't get.

The AI can not have a goal unless we somehow program that into it. If we don't, then the question is why would it choose any one goal over any other?

It doesn't have a goal because any "goal" is as good as any other, to it.

Now some AI-machines do have a goal because people have programmed that goal into them. Consider the drones flying in Ukraine. They can and probably do or at least will soon use AI to kill people.

But such AI is still just a machine, it does not have a will of its own. It is simply a tool used by people who programmed it to do its killing. It's not the AI we must fear, it's the people.

View on HN

> It doesn't have a goal because any "goal" is as good as any other, to it.

Paraphrasing the old koan about randomizing neural network weights: it does have a goal. You just don't know what that goal is.

> It's not the AI we must fear, it's the people.

Consider that AI in any form will somewhat be a reflection of ourselves. As AI becomes more powerful, it essentially will magnify the best and worst of humanity.

So yes, when we consider the dangers of AI, what we actually need to consider is what is the worst we might consider doing to ourselves.

Right. People who are worried about AI taking over the world, should rather be worried about nuclear weapons proliferation.

But I agree, AI may need some regulation, just like guns and ammunitions do.

I don't think it can be regulated, except to the extent we ensure state governments and oligopolies retain total control.

AI harm goes far beyond nuclear weapons in so much as it's capacity for harm contains everything for which we place under its control. Based on the potential direction advocates are pushing towards, that includes all of society.

It is just that its capacity for harm will be from harm it already learns from humans, or that humans purposely inject into the system for nefarious reasons, or the simple failure of humans to comprehend potential failures of complex systems.

FYI, I've written in far more detail on these topics here - https://dakara.substack.com/p/ai-and-the-end-to-all-things

> The AI can not have a goal unless we somehow program that into it.

I am pretty sure that's not how modern AI works. We don't tell it what to do, we give it a shitload of training data and let it figure out the rules on its own.

> If we don't, then the question is why would it choose any one goal over any other?

Just because we don't know the answer to this question yet doesn't mean we should assume the answer is "it won't".

It figures out "rules" within a guided set of parameters. So yes it is given direction by constructing a type of feedback on a task that it is given.

Modern AI works by maximizing the correctness score of an answer. That's the goal.

It does not maximize its chances of survival. It does not maximize the count of its offspring. Just the correctness score.

We have taught these systems that "human-like" responses are correct. That's why you feel like talking to an intelligent being, the models are good at maximizing the "human-likeness" of their responses.

But under the hood it's a markov chain. A very sophisticated markov chain, with lots of bling. Sure, when talking to investors, it's the second coming of sliced bread. But come on.

> Modern AI works by maximizing the correctness score of an answer. That's the goal.

Right. But whose goal? I would say that is the goal of the programmers who program the AI. The AI program itself, doesn't have a "goal" it would be trying reach. It just reacts base don its markov-chain.

The current chatbot AI is reactive, not pro-active. It reacts to what you type.

The correctness score is maximized by faithfully imitating humans. Humans do have goals.

They are not imitating humans in general. They are imitating the statistical average of many human written texts. That is not the same thing as imitating the goals of humans.

By imitating the speech it may look like the AI has some goal-oriented behavior, but it only looks that way. And that is precisely the goal of their programmers, to make it look like the AI has some goals.

It would be possible to have a different type of AI which actually decides on its own goals and then infers what are the best actions to take to reach those goals. Such an AI would have goals yes. But language models do not. They are not scored based on did they reach any specific goal with any specific interaction. They have no specific goals.

The only goal (of the programmers who wrote the AI) is to fool the humans into thinking they are interacting with some entity which has goals. and intelligence.

Reply on news.ycombinator.com