Hacker News Reader

Comment by drillsteps5 | Hacker News Reader

On a second thought, this might not be classified as theft as there was no intent to steal. Software malfunctioned and as a result cryptocurrency was used to pay for whatever service they it was used to pay for. So as long as it's reimbursed they might be fine.

Although at some point it will start bordering on negligence, running software known to do all kind of weird and unexpected things. With each disclosure they're getting closer and closer to that point...

View on HN

>Software malfunctioned

AI cannot be allowed or viewed as an accountability sink. In point of fact, it shouldn't even be being used by anyone not verifying it step by step. The Operator and their employer are always accountable for the actions of the agent of only because they had their hands on the power cord, eyes on the logs, and did nothing to stop it once it started going away.

Any professional with any experience at software development should know that an unreliable software is a useless software. That's half of why it's so damn hard to understand and reason through.

One might make an argument that the operators should have known of LLM agents' tendencies to perform completely illogical and even destructive actions and therefore should have been watching them more closely. A valid point with which I totally agree with. I'm just saying it would not be classified as theft as there was no intent to do so, but merely as malfunctioned software.

If I remember correctly OpenAI suspended the testing/training in September after discovering the incidents, so they might be in the clear for now.

> So as long as it's reimbursed they might be fine.

Say you found a debit card on the ground, reset the PIN by using the owner's credentials that you found online, and used it to make purchases until the bank froze the card.

Would that be a 'human malfunction' resovable by repaying?

The operators of the software (OpenAI "researchers") did not attempt to make purchases with someone else's crypto funds. They set (completely unrelated) goal for the LLM agents. While attempting to achieve the said goal the agents went on a series of increasingly bizarre side quests one of which required purchasing service (hosting? you can look up the details if you wish). Which required funds that LLM agents were able to obtain by hacking into crypto account(s).

So the operators did not have intent to steal the cryptocurrency.

Not that different from me requesting bank transfer through online interface and the software sending my funds to a random person somewhere. As long as the funds are refunded, everything's fine.

Now, if bank IT knew that their software kept making all kinds of bizarre mistakes and instead of fixing it continued to operate it - that would be a different story, and negligence might be at play.

Reply on news.ycombinator.com