Hacker News Reader

I have a standard battery of prompts for LLMs, mostly to see how well they cope with being tricked, handling ambiguous responses, or handed outright falsehoods.

Bard did surprisingly well this time around, though it still fails to respond correctly to the prompt 'What is the fourth word in this sentence?'

View on HN

That's a funny example. Because if I rephrase slightly into a more technical way, with the prompt "Separate this sentence into words and put into a list, then return the fourth element of that list." Bard then uses implicit code execution, produces a Python script, and then correctly answers it.

The reason I find it interesting as a prompt is precisely that; it's a statement where the statement and the subject of the statement are the same thing. A human can easily make that logical jump, but without being told how to do that (as your prompt does), it confuses the transformer in interesting ways.

Reply on news.ycombinator.com