One thing I always like about coding is that when something is annoying or hard to do, you can improve things so that it becomes easier and more straightforward, at least in principle. For example if a library is badly designed, buggy, or just doesn’t exist, you can write your own. If tooling is bad, you can improve it or create your own tooling. At the extreme, you can create your own programming language, database or operating system. In the small, you can write your own macros and wrapper functions.

With LLMs, that route largely doesn’t exist. Yes, you could write your own harness and agentic tooling. Maybe you could RLHF an existing model. But it all feels less tangible, and you can’t really change the interface to the LLM on the semantic content level in a well-defined way, like you can precisely control the semantics of a library. What you’re prompting against is not malleable in the manner that what you’re coding against is malleable. If your LLM repeatedly goes down the wrong paths, you can’t fix that once and for all, like you can fix a misbehaving library.