Fourth graders typically don't have access to Python for their homework assignments. To be fair to the kids, I tried it first without Python: Opus 4.6 (Feb 2026) with default Medium effort. https://claude.ai/share/1533a3e4-6757-4614-b95d-0743350a6598
pastebin of the reasoning section (no Python): https://pastebin.com/zZeG5ZnJ
It got questions 2 (Shop D) and 5 (280) wrong. It got question 3 right but the work it showed has the numbers for each shop wrong. My fourth grade teacher would have taken off points for that (shout out Mrs. Van Bladel).
Here it is again with a prompted nudge to use Python: https://claude.ai/share/e1265efb-0988-40ac-90ac-c76225b67e98
pastebin of the reasoning section (with Python): https://pastebin.com/KsP0xxZL
This time it used Python to "check its work", and answered the same questions incorrectly (2 and 5). To the model's credit, it did show the correct work on answer 3 this time.
If you look at the working of your non-Python example it gets most of the counts wrong - identifying shop A as two full notebooks plus one half notebook when it's actually three full notebooks, for example. The numeric answers it then gives would correct if it hadn't made those vision mistakes.
I've been testing vision LLMs on counting the number of pelicans in a photo for a while, they're very unreliable at that.
The best I've seen is Google Gemini 2.5 if you have it output image segmentation masks (a feature they have not included in the Gemini 3 series yet): https://simonwillison.net/2025/Apr/18/gemini-image-segmentat... - but that requires additional harness engineering, you need to explicitly cause it to use its image segmentation mechanism.
†: think "bo's'n"
Sure.
"8 7 6 5 4 3 2 1 - add minus signs and parenthesis to get 31."
P.S. There is an answer online and some LLMs will just copy it verbatim. This doesn't count.
Are you sure this is 4th grade level?
https://aistudio.google.com/app/prompts?state=%7B%22ids%22:%...
Though this is obviously not something I would expect a 4th grader to solve.
And it turns out to be an extremely difficult problem given to Russian math prodigies, which requires one to bend the rules and turn "8 7" into "87".
This is a counterexample to your argument, not evidence for your claim. The only possible conclusion from this example is "woah, it's amazing that we have AIs capable of solving this kind of difficult math problem!", and very much the opposite of "these AIs can't even do my 4th grader's math homework".