Hacker News Reader

Comment by Der_Einzige | Hacker News Reader

You need filter assisted decoding to make that possible in all cases (as in, working for both models all the time provably) I'm surprised it worked in chatGPT this time, but it won't work all the time due to lack of lexical, semantic, or phonetic info in the tokenizer.

https://paperswithcode.com/paper/most-language-models-can-be...

View on HN

It works in GPT-4 (but not GPT-3.5) even without a special tokenizer: Some problems simply solve themselves with increasing scale. But Bard uses a smaller LaMDA model to reduce inference cost. Quote [1] from Sundar Pichai in February:

> We’re releasing it initially with our lightweight model version of LaMDA. This much smaller model requires significantly less computing power, enabling us to scale to more users, allowing for more feedback.

[1] https://blog.google/technology/ai/bard-google-ai-search-upda...

I don't think it works reliably over 100 invokations of the prompt. The paper tries to get provable situations where 0 ignored constraints happen/are possible.

I'm impressed that it is working more often than not, but we need provable guarantees.

If you ask 100 humans the same thing, you'll get some mistakes too. I don't think the point is whether it makes mistakes. The Chat GPT response shown above is clearly more interesting and impressive / displays something more seemingly like intelligence than the Bard response provided. At some point scaling these things up enough becomes indistinguishable from intelligence, and at that point maybe it is.

> If you ask 100 humans the same thing, you'll get some mistakes too

Yes, still, it's pretty easy to find a somewhat good judgement human who never does that mistake. If you tell him/her: "This is of utmost importance: ..." — But as of now, these text gen machine learning programs don't totally understand that?

Reply on news.ycombinator.com