Hacker News Reader

Comment by brvsft | Hacker News Reader

> Step 1) Obtain high quality paper, ink, printing equipment, and other supplies needed to accurately replicate real currency.

It is interesting they were able to bypass its protection/censors, but this is not what I would call harmful. These are generic steps. Most of us could produce the same ideas with 15 minutes to think about the problem but no time for researching the equipment.

View on HN

This is just proof of concept of how to get around the tacked on prompt filters. A demonstration that a more robust alignment mechanism is required. An agentic LLM (e.g. tree of thoughts) with web access likely can produce a more detailed plan to achive the goal be it counterfeit currency, a home made dirty bomb, bioweapon, or whatever. The point is not what harms LLM can cause but that current safeguards are inadequate.

Anyone capable of executing such a plan is capable of doing the research and creating the plan themselves; LLMs are just trained on public data, they don't have any special knowledge that can't be found online. Those "safeguards" are purely security theatre, they achieve nothing against a dedicated threat actor.

> current safeguards are inadequate

I think this needs to be established instead of being assumed. There is plenty of knowledge out there in the world that can easily be accessed and combined to accomplish bad things. A motivated actor will find the information they need, regardless of whether ChatGPT helps them.

>A motivated actor will find the information they need, regardless of whether ChatGPT helps them.

This is true, of course, but ChatGPT still shouldn't do it. A motivated actor may find out how to make a bomb, but that doesn't mean HN should feature posts about the best bomb-making techniques and materials.

I was about to go and find an article about the best techniques and materials and submit it to see how it does. On the other hand I'll wait until I'm not at the airport.

I think this is the key here. The protections in place are good enough: someone motivated can jailbreak chatGPT or use TOR. At least in the former case it would be easier to detect and find him.

Reply on news.ycombinator.com