Same experience with code: Github Copilot is 90% right 100% of the time. Its suggestions for comments and docstrings are so bad I would never accept them in a code review, basically your old "a++ # adds 1 to a"; the generated code is always wrong, and the bigger the chunk of code generated the more wrong it is. It's kind of OK as a replacement for Ctrl+C/Ctrl+V and for generating boilerplate... which shouldn't be there in good code anyway.
I suspect that for every impressive output we see there are tens of thousands of trash outputs someone had to wade through. Further, the less popular your prompt and the smaller the representation of what you want in the model, the harder it is to get something even remotely resembling what you want. Though, I'm not a "prompt engineer" (WTF is that...), so maybe I'm just using it wrong...
Sitting around on our asses going "We don't have AGI/ASI yet, nothing to worry about" is kinda like saying 'nuclear bombs are nothing to worry about' in 1940 while watching your neighbor create an every larger pile of uranium. To think we're not going to accomplish this eventually is foolishness, and when we've already accomplished it, it's going to be very difficult to setup a legal framework that reigns in corporate interests around it.