Hacker News Reader

Comment by ryao | Hacker News Reader

This is analogous to saying a computer can be used to do bad things if it is loaded with the right software. Coincidentally, people do load computers with the right software to do bad things, yet people are overwhelmingly opposed to measures that would stifle such things.

If you hook up a chat bot to a chat interface, or add tool use, it is probable that it will eventually output something that it should not and that output will cause a problem. Preventing that is an unsolved problem, just as preventing people from abusing computers is an unsolved problem.

View on HN

> This is analogous to saying a computer can be used to do bad things if it is loaded with the right software.

It's really not. Parent's examples are all out-of-the-box behavior.

As the runtime of any program approaches infinity, the probability of the program behaving in an undesired manner approaches 1.

That is not universally true. The yes program is a counter example:

https://www.man7.org/linux/man-pages/man1/yes.1.html

Devil's advocate:

(1) Execute yes (with or without arguments, whatever you desire).

(2) Let the program run as long as you desire.

(3) When you stop desiring the program to spit out your argument,

(4) Stop the program.

Between (3) and (4) some time must pass. During this time the program is behaving in an undesired way. Ergo, yes is not a counter example of the GP's claim.

I upvoted your reply for its clever (ab)use of ambiguity to say otherwise to a fairly open and shut case.

That said, I suspect the other person was actually agreeing with me, and tried to state that software incorporating LLMs would eventually malfunction by stating that this is true for all software. The yes program was an obvious counter example. It is almost certain that all LLMs will eventually generate some output that is undesired given that it is determining the next token to output based on probabilities. I say almost only because I do not know how to prove the conjecture. There is also some ambiguity in what is a LLM, as the first L means large and nobody has made a precise definition of what is large. If you look at literature from several years ago, you will find people saying 100 million parameters is large, while some people these days will refuse to use the term LLM to describe a model of that size.

Thanks, it was definitely tongue-in-cheek. I agree with you on both counts.

The society has accepted that computers bring more benefit than harm, but LLMs could still get pushback due to bad PR.

Reply on news.ycombinator.com