Hacker News Reader

Comment by lyu07282 | Hacker News Reader

We will never know for sure, it is equally likely they did some cost savings which caused a reduction in quality. I certainly noticed that too, but we have no way to prove that, any proof can always be dismissed easily. For example I see it generate code with hallucinated variables quite often now, that never happened before to that degree, but I'm just as well part of the group hallucinating so easy to dismiss. Anecdotal evidence is useless.

We also can never independently evaluate it, OpenAI could cache messages, fine tune on public test sets, etc. etc.

View on HN

This is a common tactic observed with toxic personality disorders. People will repeatedly ask for examples knowing they can dispute any example given because the topic is subjective. You can spot the loop in these threads. An army of people comment/flag asking for examples regardless of how many are given in the thread. When you provide them examples they nitpick and call your prompting bad. Not saying it's bots but there's a pattern with these "OpenAI nerfed" threads across social media right now.

> We will never know for sure, it is equally likely they did some cost savings which caused a reduction in quality.

That is entirely not equally likely, and would be completely unprecedented, at the frontier of an emerging technology that people are pumping the money and the future of the world into to win.

that sounds very naive to me, you think the "future of the world" matters to corporations making the business decision to save money and increase profit short-term? That idea is so alien to me we might as well live on a different planet.

If you take the phrase "future of the world" to mean something positive in this context, you might be the naive one.

Aggressive quantization by itself could explain the quality difference.

Combined with stricter guardrails, I would certainly expect intelligence to go down.

Check out the tikz unicorn drawing example from the paper "Sparks of Artificial General Intelligence: Early experiments with GPT-4" (pdf page 7)

https://arxiv.org/pdf/2303.12712.pdf

No matter how much money they pump into it there is a finite number of GPUs. Money can’t make GPUs appear out of thin air. If they’re faced with the choice of lowering quality slightly or turning away new customers, it shouldn’t be surprising if they choose lowering quality.

A technology that is expensive to run and hard to scale. They’re doing work trying to scale, in what world is this unprecedented?

The part that is unprecedented would be giving up an edge in a battle that will win you the world if you win the battle, by saving a few bucks. At the highest level (think OpenAI/Microsoft, Google) money is not going to be the lynchpin for a long, long time. This thing is too close to "forever good enough" at way too many things to lose your edge by being too clever by half.

Reply on news.ycombinator.com