Hacker News Reader

Comment by chaxor | Hacker News Reader

I don't understand how everyone keeps making this mistake over and over. They explicitly just said "in 5-10 years".

So many people continually use arguments that revolve around 'I used it once and it wasn't the best and/or me things up', and imply that this will always be the case.

There are many solutions already for knowledge editing, there are many solutions for improving performance, and there will very likely continue to be many improvements across the board for this.

It took ~5 years from when people in the NLP literature noticed BERT and knew the powerful applications that were coming, until the public at large was aware of the developments via ChatGPT. It may take another 5 before the public sees the developments happening now in the literature hit something in a companies web UI.

View on HN

> It took ~5 years from when people in the NLP literature noticed BERT and knew the powerful applications that were coming, until the public at large was aware of the developments via ChatGPT. It may take another 5 before the public sees the developments happening now in the literature hit something in a companies web UI.

It also may take 10, 20, 50, or 100 years. Or it may never actually happen. Or it may happen next month.

The issue with predicting technological advances is that no one knows how long it'll take to solve a problem until it's actually solved. The tech world is full of seemingly promising technologies that never actually materialized.

Which isn't to say that generative AI won't improve. It probably will. But until those improvements actually arrive, we don't know what those improvements will be, or how long it'll take. Which ultimately means that we can only judge generative AI based on what's actually available. Anything else is just guesswork.

I'm concerned that until they do improve, we're in a weird place. For example, if you were 16, would you go an invest a bunch of time and money to study law with the prospect of this hanging of your future? Same for radiology, would you go study that now Geoffrey Hinton has proclaimed the death of radiologists in 3 years or whatever? Photography and filmography ?

My concern is we're going to get to a place where we think the machines can just take over all important professions, but they're not quite there yet, however people don't bother learning those professions because they're a career dead end and then we just end up with a skill shortage and mediocre services, when something goes wrong, you just have to trust "the machine" was correct.

How do we avoid this? Almost like we need government funded "career insurance" or something like this.

I'm not so sure that truth and trustability is something we can just hand-wave away as something they'll sort out in just a few more years. I don't think a complex concept like whether or not something is actually true can be just tacked onto models whose core function is to generate what they think the next word of a body of text is most likely to be.

on the other hand the rate of change isn't constant and there isn't a guarantee that the incredible progress in the past ~2 years in the LLM/diffusion/"AI" space will continue. As an example, take computer gaming graphics; compare the evolution between Wolfenstein 3D (1992) and Quake 3 Arena (1999), which is an absolute quantum leap. Now compare Resident Evil 7 (2017) and Alan Wake 2 (2023) and it's an improvement but nowhere near the same scale.

We've already seen a fair bit of stagnation in the past year as ChatGPT gets progressively worse as the company is more focusing on neutering results to limit its exposure to legal liability.

Yes again, it's very strange to see a simple focus on one particular instance from one particular company to represent the entire idea of technology in general.

If windows 11 is far worse in many metrics than windows XP or Linux, does that mean that technology is useless?

It's one instance of something with a very particular vision being imposed. Windows 11 being slow due to reporting several GB of user data in the first few minutes of interaction with the system does not mean that all new OS are slow. Similarly, some older tech in a web UI (ChatGPT) for genAI producing non-physical data does not mean that all multimodal models will produce data unsupported by physics. Many works have already shown a good portion of the problems in GPTs can be fixed with different methods stemming from rome, rl-sr, sheavNNs, etc.

My point isn't even that certain capabilities may get better in the future, but rather that they already are better now, just not integrated into certain models.

>ChatGPT gets progressively worse

https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboar..., In blinded human comparisons, newer models perform better than older ones.

That website doesn't load for me but anyone who uses ChatGPT semi regularly can see that it's getting steadily worse if you ever ask for anything that begins to border risque. It has even refused to provide me with things like bolt torque specs because of risk.

It could be a bias, that's why we do blinded comparisons for a more accurate rating. If we have to consider my opinion, since I use it often, then no, it hasn't gotten worse over time.

Well I can't load that website so I can't assess their methodology. But I am telling you it is objectively worse for me now. Many others report the same.

Edit - the website finally loaded for me and while their methodology is listed, the actual prompts they use are not. The only example prompt is "correct grammar: I are happy". Which doesn't do anything at all to assess what we're talking about, which is ChatGPT's inability to deal with subjects which are "risky" (where "risky" is defined as "Americans think it's icky to talk about").

There is no selected prompt, humans ask the models (blindly) some questions in a chat and then select the best one for them.

Worse is really subjective. More limited functionality with a specific set of topics? Sure. More difficult to trick to get around said topic bans? Sure.

Worse overall? You can use chatgpt 4 and 3.5 side by side and see an obvious difference.

Your specific example seems fairly reasonable. Is there liability in saying x bolt can handle y torque if that ended up not being true? I don't know. What is that bolt causes an accident and someone dies? I'm sure a lawyer could argue that case if ChatGPT gave a bad answer.

Reply on news.ycombinator.com