I don't think that doing the math by hand to simulate a machine intelligence makes it not intelligent, any more than doing a simulation of all the electrochemical signals in a human brain by hand would make a human not intelligent. Aside from the infeasible amount of time it would take, it's the same thing.

As for autonomy, LLMs don't have autonomy by themselves. But they can be pretty easily combined with other systems, connected to the outside world, in a way that seems pretty darned autonomous to me. (https://github.com/Significant-Gravitas/Auto-GPT). And that's basically a duct-tape-and-string version. Given how new these LLMs are, it's likely we're barely scratching the surface.