I think you're going about it backwards. You don't take a tool, and then try to figure out what to do with it. You take a problem, and then figure out which tool you can use to solve it.
Me personally, I only used LLM for one "serious" application: I used GPT-3.5Turbo for transforming unstructured text into JSON; it was basically just ad-hoc Node.js script that called API (prompt was few examples of input-output pairs), and then it did some checks (these checks usually failed only because GPT also corrected misspellings). It would take me weeks to do it manually, but with the help of GPT it was few hours (writing of the script + I made a lot of misspellings so the script stopped a lot). But I cannot imagine anything more complex.
Since you seem to have not noticed my comment above, here's another example of a project that implements many of these techniques. Me and many others have used this to transcribe hour long videos into a well organized "docs site" that makes the content easy to read.
Example: https://matadoc.vercel.app/
This was completely auto-generated in a few minutes. The author of the library reviewed it and said that it's nearly 100% correct and people in the company where it was built rely on these docs.
Tell me how long it would take you to write these docs. I'm really confused where your dismissive mentality is coming from in the face of what I think is overwhelming evidence to the contrary. I'm happy to provide example after example after example. I'm sorry, but you are utterly, completely wrong in your conclusions.
What you're describing is more about reasoning abilities - that's not really what the article was about or the problems the techniques are for. The techniques in article are more for stuff like Q&A, classification, summarization, etc.