Someone gives you a hammer. You've never seen one before. They tell you it's a great new tool with so many ways to use it. So you hook a bag on both ends and use it to carry your groceries home.
You hear lots of people are using their own hammers to make furniture and fix things around the home.
Your response is "I accept what you say about the best way to use these hammers. But my worry is that there is nothing that requires people to use them in that way."
These things are not intelligent. They're just tools. If you don't use a guide with your band saw, you aren't going to get straight cuts. If you want straight cuts from your AI, you need the right structure around it to keep it on track.
Incidentally, those structures are also the sorts of things that greatly benefit human programmers.