My anecdotal experience is similar. For any important or hard technical questions relevant to anything I do, the LLM results are consistently trash. And if you are an expert in the domain you can’t not notice this.

On the other hand, for trivial technical problems with well known solutions, LLMs are great. But those are in many senses the low value problems; you can throw human bodies against that question cheaply. And honestly, before Google results became total rubbish, you could just Google it.

I try to use LLMs for various purposes. In almost all cases where I bother to use them, which are usually subject matters I care about, the results are poorer than I can quickly produce myself because I care enough to be semi-competent at it.

I can sort of understand the kinds of roles that LLMs might replace in the next few years, but there are many roles where it isn’t even close. They are useless in domains with minimal training data.