Hacker News Reader

Comment by epihelix | Hacker News Reader

To be fair, that's a pebkac problem. You got exactly what you requested (and the language was unambiguous).

This is like writing `if (cond_x && cond_y) {}` and then blaming the compiler because it didn't realise that actually meant to write `if (cond_x || cond_y) {}`.

View on HN

That's not what they said, they said it limited its output to 2 issues because it saw the "both" in the sentence, which is clearly an incorrect reading of their instructions.

Yes, I think I misread -- looks like the pebkac problem was at my desk instead :/

But I don't think their prompt as written would have limited the responses to two, unless there really were only two matching both conditionals. I've not seen an LLM misinterpret language that way before.

You're right, but I do wonder if they're misremembering their prompt a bit. The AI's response would be much more likely to arise if the prompt was something closer to "find both issues that are critical and related to X". Based on my own experience, it would be a very surprising mistake for the AI to have made if the prompt really was as straightforward and precise as "find issues that are both critical and related to X". I know it's a bit annoying to question someone's account of their own experience, and obviously I'm not saying this couldn't have happened, but my guess would be that the actual prompt was at least somewhat ambiguous.

The language is not unambiguous; it's exactly ambiguous (you and your first reply demonstrate how hard it can be to see the other reading):

0. "find issues that are both critical and related to X"

...could be read as "issues that are X and are Y":

1. [valid, intended] find [all] issues that are[, at the same time,] 1) critical; and 2) related to X

...or as "issues, both of which issues are X and are Y":

2. [valid, your reading] find [an to-be-specified number of] issues that are, both [so I want two issues], possessing X and Y attributes.

Reply on news.ycombinator.com