That said, I found myself doing little text mangling tasks with GPT-4 instead of appropriate command line tools, because fuck if I remember the flags to sort and unique, and it's much faster to just describe what I want and have the LLM take a crack at it.
This is why, I think, you'll see a lot of simple(ish) tasks being done by LLMs, at couple orders of magnitude more compute cost - it's just that much more convenient.
EDIT: nevermind. That was about just doing regex replace.
But if you want to do it correctly in an automated way, LLMs are indeed the best tool for the job in the general case, because they understand how language work, in a way that you can't really formalize in code. No, feeding the complete definition of English grammar to the computer won't help, because a) it probably doesn't exist, and b) even if it does, it's merely a suggestion - natural language is a living thing, and is not bound by fixed rulesets.
"Oh, I live near there too! House or flat?"
Stanley hesitated. These questions were getting more and more personal. Was this her idea of casual conversation? Or was she trying to get to know him personally? Well, he thought, what could go wrong if I treat this like a conversation. "Flat," he said. And then asked a question of his own. "What kind of pop do you like?"
How are you going to decide if "flat" is talking about non-fizzy pop/soda, or a dwelling space with a regex? It's a lot closer to the word "pop."
Of course, GPT handles this perfectly https://chat.openai.com/share/fb6a56a2-749c-4e71-9506-541bb8...
But I'll admit I agree that this is well beyond the scope of regex.