Well, that assumes that if you just keep throwing more data and compute at large language models you'll end up with something akin to AGI to control those robots. Which is far from guaranteed.
So why do you suddenly think you need it for controlling a body when animals do it with far less?