Hacker News Reader

Comment by Wowfunhappy | Hacker News Reader

I don't understand the purpose of ChatGPT's safeguards. "Give me ideas for crimes to do?" Nobody needs an AI for that! Humans are plenty good at coming up with crimes on our own, thank you very much.

The primary danger of a tool like ChatGPT is that it will be used to build fake consensus around a topic, or perpetuate disinformation, via AI-controlled bot accounts or massive content farms. None of the measures OpenAI has put into place will combat this. If I wanted ChatGPT to tell me why Ukraine should surrender to Russia, I could make it do that. That's a perfectly legitimate and reasonable political stance—albeit also one which Russia might be interested in perpetuating.

The danger is not in the type of content ChatGPT can generate, but the sheer quantity of what it could create—and the fact that it really is indistinguishable from a human.

View on HN

Humans are perfectly capable of coming up with crimes, but in good news, they’re bad at planning crimes that won’t get caught.

It’s not one of the biggest things I’d worry about in the world (in particular, “dumb enough to need a chat bot’s help, but smart enough to get a chat bot’s help” might be a small group), but putting safeguards in place to prevent chatbots from being crime coaches seems like a good thing to try.

Edit: probably even more important, as far as the actual reason it exists, is that it’s a PR thing. Regardless of the actual harm, “the chatbot told me how to make meth” looks bad.

That's a good point.

The flip side, however, is that OpenGPT is quite severely restricting what its AI can be used for, particularly if they do want to prevent these trivial "workarounds". Stories do in fact have villains who commit crimes, and an author dealing with writer's block might turn to an AI for inspiration.

I'm not against restrictions if the benefits outweigh the harms, but the balance here seems pretty out of whack.

> Humans are perfectly capable of coming up with crimes, but in good news, they’re bad at planning crimes that won’t get caught.

How do we know this? We only know of the failed attempts, there's really no way to track how many crimes are successful outside of cases where there's obvious evidence (like a dead body).

> but the sheer quantity of what it could create

GPT models are not really creative.

The models interpolate within the existing corpus of written material, and regurgitate it. More like a search engine returning snippets of text that overlap, grammatically correct.

Sometimes the word salad dishes up a mixture that inspires you, similar to improv prompts, but it is ones own intelligence that is actually creating meaning from the generated words.

I do admit it looks creative to me, but I suspect part of that is that I admire smart surrealist thinkers, the mashing of ideas, free associating using connections that are not semantic (for example puns and many jokes depend upon non-semantic connections?).

Have you had a chance to play with ChatGPT specifically? I would have said exactly the same about GPT-3, but ChatGPT has completely blown me away. The writing it produces no longer has that surreal quality, it can create cohesive narratives.

I can't speak to how it works at a technical level, but I'd say the results are on par with what a strong middle school student might write in terms of content (and above that in grammar and writing craft).

Yeah, I have been playing with ChatGPT, and it is amazing and lots of fun.

However, it mimics patterns it finds in the corpus of text it has been taught on. However it lacks basic understanding of the semantics, the meaning and connections, behind the words it mashes together. Give it a long prompt and ask it to write a story where John lisps and Suzy barks, and it doesn’t understand what John or Suzy really represent. It has a superficial understanding of words, like an alien that has read a dictionary and thesaurus.

Note: My text above scored 99.94% “real” using https://huggingface.co/openai-detector/ and I didn’t edit it to improve the score. Your comment scored 97.81% real (admittedly the detector doesn’t work very well on short texts. I tried removing some cliches from your comment but I failed to improve the rating).

The purpose is to prevent a headline reading "Person robs store because ChatGPT told him to".

Also any groundwork for ethics regarding AI is good as far as I am concerned.

Reply on news.ycombinator.com