People using AI to generate crap faster is definitely a (short term) risk. And, BTW, not only with code. I read an article stating that in a few years (2-3, can't remember) 80% of content only will be generated by AI. Which would be a disaster. We're already swimming in low quality information and this will only make it worse.

OTOH, so much for the idea that people will guide the hand of AI to create better code. As this will unfold, there will be ever more incentives to remove (most) humans from the loop. And if past can be used to predict the future, what we have seen so far is that when AI gets reasonably competent at something it will gain superhuman capabilities very quickly. I still keep bringing up how people thought that after alpha go beat Fan Hui (European go champion) it would have taken a huge leap for it to beat Lee Sedol. Because Lee Sedol was so much better than Fan Hui. At least in human terms. It only took a few months of training and tinkering for DeepMind.

Speaking of loops: since AIs are taught from the internet (simple language models or coding specialists), we're creating an unwanted feedback loop here. More AI generated content will likely make teaching future AIs harder. At least with information harvested from the internet after ~2022-ish.