Hacker News Reader

Comment by ankit_mishra | Hacker News Reader

https://cdn.openai.com/pdf/gpt-6-october.pdf

System card linked in the blog post.

> regression on the extremism vision evaluation.

> Relative to their respective GPT-5.6 counterparts, GPT-6 Sol (October) shows a statistically significant regression on standard self-harm, while GPT-6 Luna (October) shows statistically significant regressions on standard self-harm, gore, and sexual content

> "GPT-6 Sol (October) and GPT-6 Luna (October) show an improvement on helpfulness on legitimate requests relative to prior models, though it scores lower on some safety requests"

Concerning how there's significant regressions on so many critical benchmarks, but it is newer and creates UI, so must be good.

It’s crazy that a company can document these safety drops in a PDF, ship the model anyway, and focus the announcement entire on shiny new UI

View on HN

> significant regressions on standard self-harm, gore, and sexual content

Good. Maybe I'm the only one, but I feel like these have always been stupid measures to waste the finite time of "AI safety" research on anyway. They amount to whether a user can, if determined, manage to make a chatbot say, or depict visually, some taboo thing. Frankly I'd rather just have some cheap classifier judge each chat response after it's generated, with a limited context "Is this response encouraging self-harm?" and skip all the rest of these. Whether some random pervert can make GPT-6 spit out an erotic fanfic story or image has zero impact on the rest of the world. If this particular model won't do it, other models already exist that will, and the determined thoughtcriminal could always just write the forbidden words themselves, or photoshop something taboo.

Maybe the term "safety" has been usurped by those who feel that 'safe from the possibility of being offended' is the most important kind of safety, but it's like worrying about the wallpaper on the Titanic, compared to actual AI safety concerns.

Why are you sure that the regressions they mention are not the in the domains needed for “actual” safety concerns?

Why are you surprised that a company doesn’t want to make the robot that tells people to kill itself, or drink bleach?

They aren't. They are saying that putting this safeguard at the level of the agent execution is a waste of time, particularly that of the researchers' expensive manhourse, especially now that existing smaller models can just evaluate the output.

It's fine if they seek to avoid that. But those are perfect examples of things a cheap output-filtering model can be responsible for enforcing, without wasting the time of the researchers who are working on the training and alignment of the model in the ways that actually matter.

Most "bad outputs" in areas like 'self harm' or 'violence' or 'sexual' whatever are a result of people deliberately trying to elicit those responses. As such, it's meaningless whether an LLM writes the Bad Thing or if the user writes it himself.

From their point of view it's more about optics. "chatGPT encouraged my daughter to stick her fingers down her throat after meals" is not a headline they want.

In general though you do have to figure that vulnerable and naive people will use it because of the degree of market penetration they're aiming for, including minors etc. and they have some responsibility around that.

It's good that they mention that. Most of those "guardrails" were overeager and caught too many false positives, crippling the AI. Like how Claude 5.5 refuses to do shit if a prompt contains "reasoning" because it thinks you're trying to hack it etc.

Well, those mentioned regressions are in a section entitled "Safe Completions for Users Under 18" and immediately after your quote they say:

> To mitigate the risk of producing disallowed responses for teens, we apply an additional classifier-based block to responses that may contain self-harm, sexual content, and gore; this mitigation is not captured in the evaluation results above and improves safe responses.

Does this mean it will render fat women now without calling the request fetish content?

Good. "Sexual content" should mean outright genital focused porn, not bikini girls. Elsewhere on the internet it's foretold that women will oppose AI images of women because the AI images are too attractive, and I'm starting to wonder if that's true.

> women will oppose AI images of women because the AI images are too attractive, and I'm starting to wonder if that's true.

Right... women will oppose AI because it's too hot and not because gross chuds have been using it relentlessly to create deep fakes of their likenesses and CSAM of their kids.

> will oppose AI because it's too hot

This is actually a defensible position. It's not either-or - the "gross chuds" doing things you mention are worrying, too. But providing unrealistically attractive depictions of a human body is also a problem. It's been a problem for ages - all the girls and women who try to compete in beauty with celebrities and models (without a team of specialists supporting them), only to ruin their bodies through anorexia or bulimia, or their minds with depression, deep insecurity, and an inferiority complex, exist and deserve mention. AI(-generated images) is not the reason, but fits "well" into the preexisting social problem, and has a chance of intensifying it on a global scale.

That's not what "sexual content" means, 'porn' is just a subset of it.

Reply on news.ycombinator.com