Hacker News Reader

Comment by moyix | Hacker News Reader

How could it be unsafe? Well, if you ask it how to deal with a grease fire and it recommends pouring water on it, that seems like it would be unsafe. Or perhaps you hooked a Python script up to the output of ChatGPT and let it call API functions that control a robot arm; that could also be unsafe. It doesn't seem like a crazy thing to worry about.

I'm not sure what you mean by "by itself" – at the very least, the comments section here is always filled with humans who say they find ChatGPT useful. I haven't personally found it very useful yet, but Copilot certainly is, and it's very similar to ChatGPT in terms of architecture.

View on HN

> if you ask it how to deal with a grease fire

This is absurd. If I ask a child how to deal with a grease fire, they wouldn't know the right answer either. Even looking stuff up on Wikipedia is wrong half the time. Toys like ChatGPT will literally never replace subject matter experts (let alone provide nuance w.r.t. ethics or law), so I'm confused why this is even the bar here.

These are valid examples of possible unsafe behavior by ChatGPT, just as you asked for. They are not absurd, you pointing out a child doesn’t know the answer provides nothing of substance to this discussion. Wikipedia is not wrong half the time unless you are only looking up culture war issues that you don’t agree with. ChatGPT doesn’t need to replace SME’s to be useful to the population.

> They are not absurd, you pointing out a child doesn’t know the answer provides nothing of substance to this discussion.

It does, because neither the child nor ChatGPT are equipped to answer that question. Knowing about how AI models are trained and how future tokens are inferred, I think it's fundamentally absurd to think that ChatGPT can consistently give you the correct answer within any reasonable margin of error.

I think the margin of error for simpler tasks is relatively low. It seems everyone is testing its boundaries right now and finding out what tasks it can be used for and others that it will fail at due to the limitations of these LLMs. I’ve found many uses cases for it in my personal life where it performs from adequate to fantastic. I’ve rarely found failure scenarios because I’m not pushing those boundaries. I would not discount the simply things that are now possible for ordinary people because of ChatGPT accessibility.

I would say that there is generally a notion that a machine should be designed in a way that minimizes harm even when it is misused, especially if it is intended to be used by the general public. I don't know if you agree with that, but I would say OpenAI seems to, at least superficially.

Reply on news.ycombinator.com