> they can just write traditional code on the fly

this is a point that i think is missed on many people (myself included earlier in the year). Sure, the LLM is non-deterministic but it can write code that is not and test cases to verify it. Then, once verified correct, it can use what it wrote to perform the task in a deterministic way. Even though the LLM is still non-deterministic all it has to get right is the arguments to the software which is much easier than the full logic of the task. That's very powerful.

I've seen this first hand when asking AI to do non-trivial analysis over a couple of spreadsheets and then also a database. It writes python to do this, tests it with sample data, and then calls the python it creates to do the processing on the actual data.