Hacker News Reader

> despite pretty much just predicting the next word

Is GPT really like the old school NLP trigram stuff?

"predicting the next word" sounds trivial until you realize the value and complexity of "predicting the next word[s] by a world leading expert in a particular domain".

Quoting famous people sounds smart until you realize they just memorized a ton of trivia. These models have demonstrated that they don't learn logical models, instead they learn to generate text that looks logical at first glance but is nonsense.

I asked GPT-4 something fairly niche that I happen to know a fair amount about: to explain the concept of Xenon poisoning in a nuclear reactor. Other than skipping Te-135 being the initial fission product that starts the decay chain (and tbf, operationally it can be skipped since the half-life is 19 seconds), it got everything correct.

I'm sure if I kept probing on smaller and smaller details it would eventually fail, but I'd argue for _most_ people, on _most_ subjects, it performs incredibly well.

I couldn't get ChatGPT to do Wordle. I don't have access to GPT-4. See if it can do that.

Ok, but it has all this information already. So why is this surprising.

A real test of it's abilities would be synthesizing a NEW type of nuclear reactor.

Reply on news.ycombinator.com