In other words: I suspect that the output is heavily derivative from online discussions, and not based on the papers.
Of course, the real proof would be to see the output for entirely new papers.
In other words: I suspect that the output is heavily derivative from online discussions, and not based on the papers.
Of course, the real proof would be to see the output for entirely new papers.
It shouldn't be surprising that a LLM is able to understand a paper, just upload one to Claude 3.5 Sonnet.
For all the other papers, assuming they were impactful, they must have been referred by others, highlighting what their contribution is, what is controversial, etc.
In other words: the LLM doesn't have to "understand" the paper; it can simply parrot what others have been saying/writing about it.
(For example: a podcast about Google Illuminate could use our brief exchange to discuss the possible merits of this technology.)