The best thing I can say about evaluating LLMs for code is that they reminded me of the infinite monkey theorem.