If the LLMs were thinking in the sense that some people say they are, they will easily recognize the adversarial nature (e.g. letter counting in the language) and build their own tools to write conformant code right away. As a normal human would also build tools to solve the things we know we're not naturally good at.
The effect should be that a few tokens are spent on building the tool, and then the rest of the problem can be solved with a "normal" amount of tokens without brute-forcing the solution.
If the LLMs are glorified pattern matchers they might not find enough patterns in the corpus of human knowledge and remain inefficient at it.
The weakness I see is rather that this kind of problem-solving can be pattern matched on the corpus of existing human knowledge, so it may be that pattern matching on human thinking might still be enough to solve these problems.
I'm antagonistic to the deployment and use of LLMs for a variety of reasons. Nevertheless "pattern match on a huge database" is a poor way to think about how these systems operate. They are networks of semantic and syntactic relationships; when you inject words, the network activates in ways that are hard to predict if you think of it mere "pattern matching" (although some of the activation will be that). If you then feedback the activation into the network (various ways to do this), the result is even less like "pattern matching" a just a little bit more like "thinking" ...