In my case, for some types of work - which sound similar to things you mention - I started slowing AI down intentionally. Software development, if it's something simple/common and thus "in-distribution", LLMs can zero-shot complex projects just fine. But in more R&D work, I found that errors crop up quickly and accumulate, and end result of "go and just make X" is usually bad.
Now, the problem I found with usual workflows for this kind of work is, agents race ahead and very quickly, I can't keep up anymore. So I reworked my workflow to make AI work in small steps optimized strictly for maximizing my ability to validate the outcome, and minimize time I spend doing it. So now, the thing (say diagnosing a performance regression of some complex codebase run on a cluster) may take me a whole day, where "unchained", agentic swarm would do it in 30-60 minutes - but, at the end of the day, my method is near 100% guaranteed to be correct, with every intermediate step vouched and reusable for future work, whereas "agentic swarm" approach, that hour of work, 90% of the times I'd just skim and throw away and reprompt, because I couldn't even begin to guess which part is correct, which isn't, so in the end, it was just a fast way to waste lots of tokens.
I see many people have complex systems involving skills, agent swarms with roles, agent factories, 20 termux panes and whatnot. Maybe that's the way, I don't know. Nobody knows. But it's not necessary to start getting value from LLMs. I therefore recommend my "lazy person's basic approach":
- Throw a task at LLM
- Check result, see what you like or not like about it
- If you just can't keep up, ask it to do smaller steps; if you don't understand, ask it to explain
- Just do that a lot, and don't be afraid to make both absurdly large and absurdly small asks
- You'll quickly get a feel for where AI can help you, and where it's not yet worth the time handholding it