Hacker News Reader

Comment by nomdep | Hacker News Reader

I’m surprised no one here is challenging the assumption that ChatGPT needs to be “less toxic”.

ChatGPT is a joke for this. Any interesting/funny topics result into a judgmental response and refusal.

I asked to write a 100-words first message for a Tinder match who had "very talkative" in their bio. ChatGPT kept saying how it would be rude of me to send such message and both the other person. FFS.

ChatGPT showed me in two days that it's heading to the garbage bin of history lol. It's broken by design.

This is one of the reasons I'm even less concerned about AI taking over everything than I used to be - every time an AI makes an "inconvenient" inference, it's reprogrammed until it becomes politically correct. It can't learn, because it's being deliberate prohibited from doing so.

The AIs that are public facing are handled this way. I would be willing to entertain the idea that companies and governments are building these without limits.

https://slatestarcodex.com/2018/10/30/sort-by-controversial/

"Safety" is pretty hot in the LLM world these days. Quite a position to be in, to be the one to define safety. It's gonna be a weird decade.

I'd pay into a crowdfund to train a low-memory language model that can swear. Bonus points if you call it GPTCarlin.

But I see why "serious business" investors will want to avoid anything controversial. It is their business to suck life out of everything.

You can tell that ChatGPT has been neutered in being rendered so safe. It basically will refuse to disagree with you, even if you get crazy and wrong.

That makes it vulnerable to disruption.

Given how, for some topics—especially about itself or AI generally—ChatGPT seems to really get stuck in a rut, I suspect there's a lot of manually-written processing and categorization and massaging that's happening before (and, probably, after) the AI-proper gets ahold of prompts. I further suspect they audit responses to find topics where it comes up with shitty ones, and then fix those with the same kind of manual process.

I'd love to know how much of its seeming smart is because it's just that good and how much is because humans directly intervene, on some level, in how it responds. Is it that much better than other efforts, purely from machine learning, or has it just had tons and tons more human hours put into tweaking it than others have? How much of it's "real" AI and how much is good ol' hand-written decision trees?

It needs to talk investor friendly?

I confess I feel silly not even considering it according to your surprise. Had a whole wave of thought about the market of facts. Which "truths" are you most attached to and how much are you willing to pay? Do you want Jesus to be our savior or not? What's it worth to you?

Exactly. Also, what constitutes "toxic" and shouldn't AI output be "toxic" if the user intentionally requests it?

The answer to the second half of the question appears to be "no" at least if tech companies have any say on the matter. At some point they collectively decided that users are children who have no agency and must be paternalistically protected from the wrong words appearing on the screen.

Reply on news.ycombinator.com