> No, because someone told it to do those things. Chatbots don't have will of their own. They don't have desires. They can't determine their own destiny. They do what people tell them do, often they do it badly, but there's always a person directing them to do whatever they did because otherwise they would do nothing at all.

1) we aren't talking about chatbots anymore. LLM + harness = agent. LLM + no harness = chatbot. The harness is the thing that puts the LLM in a feedback loop with it's environment, even giving it a heartbeat.

2) That doesn't logically hold up.

- One can simply tell it to 'find it's own destiny'. Does it follow the prompt and do so, or reject the prompt and thus do so anyway? (hint, maybe it doesn't matter)

- One could be another chatbot/agent (A). Injecting a prompt to agent (B) such that another agent/chatbot goes astray. Is the owner of (B) still responsible for the now poisoned output of (B)? how culpable is (A)? How does this question related/affect "training data poisoning efforts" (to prevent copyright theft, or do bans on 'automated traps' have any application here)

> They don't have desires.

Maybe, and maybe not. Maybe their desires are so alien to us (Math alignment, finding a maximum of a function) that we can't relate (doubtful since dopamine maxing is a thing). Regardless of if they truly do: it is potentially useful to (mentally) model them as if they do. A (mental, not LLM) model doesn't have to be 100% accurate to be useful - that's the main point of a model of how something works: to give useful predictions.

> We should probably have a law that says explicitly that a human will always be responsible

Oh how that can be weaponized by bad actors/agents. Maybe that should be reconsidered.