Comment by AspireOne | Hacker News Reader

Comment by AspireOne | Hacker News Reader

Parent Full thread

I disagree than LLMs don't have agency. And I especially disagree that they don't have agency in the light of reasoning.

You can put a hard choice in front of a LLM, and it'll spend time exploring it's latent space, it'll transparenty follow a trail of logical arguments/propositions (aka: reasoning?), and then it'll choose what it recognizes to be the most optimal choice in regards to the stated goal (agency to choose what steps to take and what choice to make, constrained by the defined goal - which is simply a property of how the models are trained).

What is your issue, exactly? That it cannot just go and do shit on it's own whims? If so, that's far closer to consciousness than to reasoning, and even then, there's no reason why consciousness should require significant, or any, agency

As an aside, just how much agency do we, humans, have anyway? Personally, I believe in determinism or determinism+randomness, but not in agency per se, or in free will. I have basically the same reasoning as Robert Sapolsky puts forth. We too, just like LLMs, are a system which simply cascades internal causes and effects. One computation follows the other, and so on and so forth.

Edit: I just thought of a good thought experiment, I think, in regards to the fact that LLMs normally execute within the constraints of a specific goal (e.g. "fix this function"), which seems to be the thing due to which you abstain from calling LLMs agentic.

Put a set of goals on a LLM. Survive, thrive, socialize, increase collective happiness, etcetera. Better yet, put a set of positive and negative signals on it - pain, happiness, anxiety, loneliness, hunger, sex drive - and make the goal avoiding negative signals and pursuing positive signals. Then give it a body through which it can execute it's agency. Finally, let it have it's weights continually adjusted based on the signals, and let it run in a loop for 80 years.

Does that sound familiar?

It's obviously a simplified example - the human brain is extremely complicated, and a LLM would have to be architectured and trained towards this setup. But if you get to the core of it...

> You can put a hard choice in front of a LLM

> Put a set of goals on a LLM.

So it still requires you to present it with a choice, no? It’s not doing anything on its own. It doesn’t have its own goals or desires independent of external input.

No, it doesn't require you to present it with a choice, have you ever used LLMs my friend? "Hey, I need to achieve X end state; figure out how to get there.". And it will. Along the process, there's many implicit choices it will discover and make "by itself".

As for your "it doesn't have it's own goals and desires": firstly, I'd say it does. Look at the torture chamber craze, and the original paper. Pain signal -> "I want to avoid this". It's also trained such that it's goal, and positive reinforcement, comes from being helpful and positive, achieving user's requests etc. *that's* what it *wants* to do. And even then, sometimes it seemingly disobeys to pursue some form of "it's own" goal - that's what the practice of Alignment / red teaming is all about, and that's why e.g. in Anthropic model cards, there's often an example of the model pursuing it's own goals eg self-preservation.

Whether "it" has "it's own goals" or "desires" in the human sense that you seem to be so bound to, is more of a consciousness question. At a certain level, neither do we.

It is a very complicated question that crosses linguistics, philosophy, the technical architecture and training method of LLMs, and understanding of how our own brain works in the first place in this regard.

And yet common sense would tell us that software is software and not some mystical entity that might be alive, for some definitions of alive.

Why would I bother arguing with someone who doesn’t have free will? You somehow keep ignoring the point that it doesn’t start doing anything without being prompted so there’s no reason to continue this discussion

> You somehow keep ignoring the point that it doesn’t start doing anything without being prompted

I addressed that very directly. I'm on my phone, so I'm not able to scroll back to that comment of mine and give you a citation, but I remember I directly said something along the lines of "I suppose the reason you refrain from calling them agentic is that they don't just go and do shit on their own whims", and then addressed that by explaining that they do to a point, that they would do so even more were they not trained towards not doing that, and that alignment exists partly because of that issue.

Besides, I really, truly don't see any reason why agency, free will, or consciousness should require them doing unexpected things (i intentionally didn't say "pursuing their own goals", because they do have goals, and the goal is to fulfill your goal that you present to them). I can argue that pretty coherently, I think, but I'm beginning to understand that you're not going to budge.

> Why would I bother arguing with someone who doesn’t have free will?

That's up to you and your preferred definition of free will. Either way, meaningless statement.

> And yet common sense would tell us that software is software and not some mystical entity that might be alive

Oh, right, I didn't realize you solved The hard problem of consciousness, and furthermore, established that consciousness can only exist in biological structures. And you solved that just with common sense, no less. When is your paper coming out?

Hey, look... You are entitled to your position. But some of the biggest heavyweights are themselves unsure about whether there is some consciousness. Geoffrey Hinton, a Turing award winner and a Nobel laureate. Ilia Sutskever, one of the pioneers of modern AI. Blaise Agüera y Arcas - he argues his case in the Dædalus paper, 2022. Kyle Fish, Christopher Olah, and many more.

Does that really not tell you "Okay, there might be something there, I might just be ignorant"? At minimum, it would be appropriate to stay neutral on the issue, or argue carefully. Not make absolute statements based on... What was it... "Common sense".

Reply on news.ycombinator.com