> What surprises me about this is how poorly general-purpose LLMs do. The best one is OpenAI o1-preview at 18%.

o1-preview doesn't even have image input, so I wonder how they used it.

Also, Ryan Greenblatts solution basically does "best of 4000" iirc. Presumably o1-preview was single shot.