I write CLI tools with LLMs all the time. I even have a custom Claude Project that teaches the LLM to use inline script dependencies with uv so I can "uv run script.py" without first having to install anything else: https://simonwillison.net/2024/Dec/19/one-shot-python-tools/
I have a collection of tools I've built in this way here: https://tools.simonwillison.net/python/