On the other hand, I find that "someone who has read all the blogs and papers and so can suggest something you might have missed" is my favorite use case for LLM, so seeing that it can miss a useful (and novel to me) idea is annoying.
For an LLM, I’d expect it to be similar. It can recall the stuff it’s seen thousands of times, but has a hard time recalling the niche/underdocumented stuff that it’s seen just a dozen times.
The human brain isn't a statistical aggregator. If you see a psychologically socking thing once in your lifetime, you might remember it even after dementia hits when you're old.
On the other hand, you pass by hundrends of shops every day and receive the data signal of their signs over and over and over, yet you remember nothing.
You remeber stuff you pay attention to (for whatever reason)