Hacker News Reader

Comment by comboy | Hacker News Reader

> It doesn't have an understanding of sippy cups.

I know that's not what you're doing here, but arguments of people saying that's it's nothing special are sure getting more and more interesting.

View on HN

I haven't touched GPT4 yet, but there are legitimately things GPT3.5 does not understand. Often simple, easy, intuitive things. Pretty much anything that isn't being optimized for as a marketable product. Ask it law questions, perfect it passes the bar. Ask it for an ascii drawing of San Fransisco (or any other text encoding of geography), it fails flat on its face often placing things labelled "East" to the left and vice versa or listing the same place twice. Ask it to make a transcript and it often messes up which speaker has which role and what is appropriate for them to say. Ask it to do any sort of recursive transcript, that is a transcript inside of which another transcript will be generated and discussed within the conversation, and it blatantly ignores you. All this and so much more.

GPT4 is much better in that regard. With GPT3.5 I've had very similar problems. Plainly ignoring very simple directions (e.g. Do not output sentences starting with "In conclusion"). GPT4 unfortunately still does that, but it's like 10% of the problem that it used to be.

I also happened to test nested contexts, and at least when going meta-meta it's doing noticeably better than it did before.

I would have never guessed such performance is achievable with this kind of token predicting text-only model.

If current trends continue, in 10 years time I’ll no longer be able to pass a Turing test.

Reply on news.ycombinator.com