Hacker News Reader

It's not 'hallucinations', it's garbage in, garbage out (GIGO)

ChatGPT is about as far removed from AGI as it's possible to get. I wouldn't worry about it at all, it's just another tool, albeit a toy one at this point.

View on HN

What about this "emergent" intelligence when size of the model is large enough? There are blogs about it being able to model problems internally or forming a mini NN of its own.

As I've understood, even the researchers in the field have no idea why this happens. It's completely empirical and a "discovered phenomenon" at this point.

[1]: https://www.jasonwei.net/blog/emergence

[2]: https://jacobbrazeal.wordpress.com/2022/09/23/gpt-3-can-find...

You have millions of tutorials that are about a few pages long, I'm pretty sure it got its programming skills from those. But we are already at model size where the model can keep an entire tutorial page in its context, there is no extra benefit here from expanding the context size, it would need similar tutorials but much much longer and expansive to get better than it is now, and there just aren't enough humans to write all of those.

Reply on news.ycombinator.com