Well the prompt is there to provide agency isn't it?
However, if you ductaped a simple goal directed reinforcement learning embodied agent onto a large language model, I think you'd approach something resembling animal-like intelligence. It would have motives, and the means to try to achieve them and adapt to it's environment. There's research in using LLMs for "internal dialog" in robots which is super interesting (https://innermonologue.github.io/)