Hacker News Reader

Comment by oceanplexian | Hacker News Reader

Yeah they definitely changed the model. In ChatGPT Pro you can actually select to use the legacy model or the new one, the new one is substantially worse at everything from coding to analyzing and categorizing text. My guess is it takes ridiculous amounts of VRAM (100’s of GB per user) to do inference so they changed it.

View on HN

What's funny it was first posted as "Fast vs Normal" rather than "Default vs Legacy", with Normal selected by default presumably because that made it better

Then overnight "Normal" became "Legacy" and "Fast" became the default.

I do wonder what is the real economic cost of running a ChatGPT query, once you peel away any introductory pricing etc. IE amortised cost of hardware and research, plus running costs (electricity).

Whenever I see the size of these models, it strikes me that it must be quite a bit, certainly not pennies.

In the all in podcast they said 30 cents / query (probably for the original version), and 1-3 cents for Google search, that’s why what Bing is doing can be so disrupting to Google margins as well when they introduce it.

I noticed this too. I'm considering stopping my subscription and claiming a refund. Chances they'll drop legacy soon and it's usefulness for me goes with it.

Someone said you can choose the legacy model of you are a subscriber, have you tried that?

Perhaps question if this will remain?

Yea I think they shrink the model size for sure

I'm guessing that new one is being used for the free tier, too, because it seems to be pretty bad for non-coding tasks too. I tried to use it for some very simple math (see comment above) and it was atrocious.

ChatGPT is a language model, and as far as I understand it isn't designed to do explicit computations or logic.

logic it is good at. math isn't logic as weird as that sounds

it's trained on code which follows logical rules

if I'm wrong here please correct me

They probably changed it from a davinci-based model to ada or curie -based model.

It was actually saying the model names in the ChatGPT Plus URLs today. text-davinci-002-something.

But we don’t know whether davinci still maps to 175B anymore.

Realistically speaking, in order to offer ChatGPT Plus, they will have to have more model instance running concurrently. Because the promise is no wait time, but they can’t magically increasing number of hosts over a short period of time. DC took months to ramp up.

So the realistic solution is to shrink model size so one instance can serve two models, 3 models or even 4 models

And label it as Turbo mode

can you expand on that last part?

Reply on news.ycombinator.com