Right, this is why I (author here) close the article mentioning that product design needs to keep the humans in the loop for these models to be useful.

If the product is designed assuming humans will turn their brain off while using it, the fundamental unreliability of LLM behavior will create problems.