Hacker News Reader

Comment by I_am_tiberius | Hacker News Reader

It would be interesting to know how this occurred. I assume there may have been last-minute high-level feedback suggesting: "We can't let users see that the new model is only slightly better than the old one. Adjust the y-axis to make the improvement appear more significant."

View on HN

Maybe they asked GPT-5 to update slides.

GPT-5 would've caught this mismatch for sure

That seemingly depends a bit on how hard you ask it to think, or how hard it decides to think based on your question.

"ChatGPT, this slide deck feels a bit luke warm, help me make a better impression"

I could completely believe someone who is all-in on the tech, working in marketing, and not really that familiar with the failure modes, using a prompt like this and just missing the bad edit.

Claude and ChatGPT actually took me several prompts to get them to identify this. They recognized from a screenshot that labeled axes that start at zero can be misleading, but missed the actual issue.

thats hilarious actually. gives credence to the gpt theory haha

How hard would it be for the model to be like: "Okay, there's a bar chart in the picture, the left bar is 350 px in size and the right bar is 120 px. Meanwhile, the labeled values are X and Y, which doesn't seem to match those relative sizes due to this math I ran thanks to all the cool deterministic tools I have."

Apparently quite a bit.

There's only one error, bar height for o3. Somehow height uses value from 4o, which seems like some sort of copy paste error.

EDIT: I was looking just at the first chart. I didn't see there's more below.

Did you miss the picture below where the bar for 50% is lower than the bar for 47.4%

And even if it’s just one chart. There are 3 or 4 bars (depends on how you count) so they screwed up 33%/25 % of the chart.

Quite an error margin.

Oh, I did miss it. Thanks!

> Did you miss the picture below where the bar for 50% is lower than the bar for 47.4%

That one was added later, if I interpret the attribution at the bottom correctly. And I'm also pretty sure it wasn't there when I first saw it pop up.

I mean this is the industry standard. For example every time Nvidia dumps a new GPU into the ether, they do the same thing. Apple with M series CPUs. They even go a step further and compare a few generations back.

It's dishonest and the multiple examples in the same presentation tell you what you need to know about the credibility of the presenters

There is a smell of desperation around OpenAI, so I wouldn't be surprised if this level of hypevibing came from the top.

The other chart on that slide was actually to scale. My suspicion is that it was super rushed to the deadline for this presentation and they maybe didn't use excel or anything automatic for the charts, so they look better, and they missed the detail due to time pressure.

Reply on news.ycombinator.com