Hacker News Reader

Comment by ceejayoz | Hacker News Reader

> How did you arrive at that answer?

This sort of question - "you don't remember what I just asked you?" - is one of the most glaring failings in Apple's Siri and I hope this leads to it being solved soon.

View on HN

That's the biggest problem with dialog AI systems - they don't do well with context. Context requires some kind of model of what is being talked about. Most systems don't have that, or they just have some predefined "slots" to fill in about what's being discussed.

Whoever really cracks this will own the customer support business.

it could be interesting to gauge how entwined the “how did you arrive at that answer” process is with the answering itself. i.e. which paths do they share? even at this early a stage: is there some structure which is used to determine the operand(s) that’s leveraged in both of these prompts? is the “how did you X” answer leveraging most of the “X” circuitry and just bailing out early? or does it deviate as early as post-tokenization?

philosophers would like to know.

My initial guess is they have nothing to do with each other. It would be like explaining why the next idea pops in your head. You can create a rational explanation but there's no way to test it.

my thoughts too, based on limited understanding of GPT. but the more pressure you apply towards compressing the neural network during training, the more circuitry these paths are likely to share. it would be interesting to see just how much and which parts could be folded together before you start to lose significant fidelity (though unfortunately the fidelity seems too low today to even try that).

Reply on news.ycombinator.com