I basically don’t outsource stuff to an LLM unless I know roughly what to expect the LLM output to look like and I’m just saving myself a bunch of typing.
“Could you make me a Go module with an API similar to archive/tar.Writer that produces a CPIO archive in the newcx format?” was an example from this project.
This pattern of LLM usage has been great for leaning the other's skill set so we can more effectively review each other's code. I can spend a week planning and iterating with Claud on TypeScript, then have my peer review and correct both the implemented outcome _and_ the plan that produced it, allowing me to learn how to drive the LLM more effectively in my non-preferred language. The same is true of him, he's able to autonomously learn and iterate on Go in a way that's efficient and respectful of my time.
More anecdotal evidence supporting the concept these tools are a super-power for experienced engineers, especially when you have a small group of them working together in multiple languages.
Much of the time, it generates basically what I would have written, but faster. Sometimes, better, because it has no concept of boredom or impatience while it produces exhaustive tests or fixes style problems. I review, test, demand refinements, and tweak a few things myself. By the end, I have a working thing and I've gotten a refresher on things anyway.