I try to have a good overall idea of what I want to build from the start. Sometimes I don't know and that involves long walks and iterative conversations with the agents. It's okay to stub out functionality or modules and the agent will be comfortable with this. As the agent builds, I have it update an architecture document.
It's always good to have another agent (sans existing context) review the project for slop and hacks, best practices, security flaws, etc. I usually have Codex do one, and Claude do the other. Hope this makes sense.