Hacker News Reader

Comment by enumjorge | Hacker News Reader

Do you have any sources for that?

How would making ChatGPT less likely to return a racist answer or hate speech affect its ability to return code? After a question has been classified into a coding problem, presumably ChatGPT servers could now continue to solve the problem as usual.

Maybe running ChatGPT is really expensive, and they nerfed in order to reign in costs. That would explain why the answers we get are less useful, across-the-board.

That may not be the reason after all, but my point is that it’s really hard to tell from the outside. There’s this narrative out there that “woke-ism” is ruining everything in tech, and I feel like some people here are being a little too eager to superimpose that narrative when we don’t really have insight into what openAI is doing.

View on HN

Maybe the problem is analogous to what Orwell describes here:

"Even a single taboo can have an all-round crippling effect upon the mind, because there is always the danger that any thought which is freely followed up may lead to the forbidden thought."

https://www.orwellfoundation.com/the-orwell-foundation/orwel...

This is what I'm talking about though. The fact that you're quoting Orwell suggests that you're having an emotional response to this topic, not a logical one. We're not talking about the human mind here. ChatGPT is not a simulation of human thought. At it's core it's statistics telling you what the answer to your question ought to look like. You're applying an observation about apples to oranges.

Why? Constraints on the reward model of LLMs restrict their generation space, so GP's quote applies

There are a lot of people who are entirely okay with the censorship but think it should be done in a different layer than the main LLM itself, as not to hurt the cognitive performance. Alignment is just fine-tuning... any type of fine tuning is possible to teach unwanted skills, and/or catastrophically forget previously learned skills. That is likely what is going on here, from what I can tell from the reading i've done into it.

Most are arguing for a specific "censorship" model on the input/output of the main LLM.

Here's the full talk[0] from a Microsoft lead researcher that worked with the early / uncensored version of GPT-4.

Simplified, tuning it for censorship heavily limits the dimensionality the model can move in to find an answer which means worse results in general for some reason.

[0]: https://www.youtube.com/watch?v=qbIk7-JPB2c

Reply on news.ycombinator.com