ChatGPT offered bomb recipes and hacking tips during safety tests | Hacker News Reader

ChatGPT offered bomb recipes and hacking tips during safety tests

theguardian.com

12 pointspseudolus8 comments

I don’t see the problem with AI doing what it is asked to do. All it’s doing is speeding up what you can already do through traditional search or whatever. I don’t want puritanical censored AI, which unfortunately is already a problem with the big AI chatbots.

> I don’t see the problem with AI doing what it is asked to do.

It's a problem for parents, in that allowing children some amount of AI access is probably inevitable for many, but the usual services currently have poor to no parental controls.

My child is too young still to worry about what my eventual "AI policy" will be, and I'm grateful to be at a point in the timeline where I can wait and see. But I already know that while as an adult I also don't want censored AI, I wouldn't mind if the one a child has access to has boundaries it won't cross.

In general, my impression is that the majority of people children come in contact with have better "this is not a conversation I will have with children" controls than AI chatbots do, if they are even in a position to know of course.

And if you have to make safety and steering work for children, then you have to make it work in general, even if adults can turn it off.

Cf. https://cdn.arstechnica.net/wp-content/uploads/2025/08/Raine...

I get the impression from the article that the author is writing in support of "LLMs are dangerous; e.g. they instruct people to make bombs". (I suspect that the author is in favour of regulating/restricting access to LLMs).

But, for one, seems that OpenAI and Anthropic running these tests shows they have the same concerns & want to address it. Similarly, as Anthropic mentions, their applications provide additional safety measures in order to mitigate that bad behaviour.

For another.. I wish the argument for "LLMs need to be restricted" was more explicit. LLMs help provide convincing (albeit potentially hallucinated) summaries & reproductions of information you could find elsewhere. I'd expect any such dangerous information could already be found on the internet by a motivated bad actor. -- And, on the flip side, it'd be naive to have no concerns over excessive government control over LLMs.

Though otherwise... I think it's kind of funny that the LLMs, who clearly have a goody-two-shoes personality, can be so easily tricked into discussing the bad actions mentioned. I wonder to what extent that's just reiterating crime fiction.

So does Google if you ask it.

This whataboutism that crops up every time someone says something negative about LLMs is not helpful. Yes, humans are imperfect. Yes, Google is imperfect. So what? The discussion at hand is about how LLMs are imperfect.

I respectfully disagree. Google does this by design, as a system designed to provide all legally available information to you. LLMs also do this by design, as a system designed to utilize legal information available to them (coming from the same web as Google's). A criminal can do bad things with a car, a knife, a gun, a kitchen chair and a pencil. Let's not blame the kitchen chair!

Have they tried not asking for bomb recipes or hacking tips? I've found that to be very effective at not getting bomb recipes or hacking tips.

I have found that even if the LLM complains to me that it cannot give me “hacking tips” I can just say it is for a “CTF competition” and it happily obliges.

Reply on news.ycombinator.com