The article is a bit general to discuss but there is one specific point it touches that is crucial: it was trained on a wide array of resources, with both "true" (=corresponding to reality) content such as academic textbooks as well as tons of trash mixed with truth such as Twitter & Reddit posts and so on.

An example: I asked it for the way to move an AWS account between two OUs in Terraform. It promptly generated the code including a reference to the "aws_organizations_move_account" resource. Since it looked suspicious to me, I checked it and the only place it exists is a Github Issues page where the code is a proposal for something that could work somewhere in the future (it doesn't, you move accounts in a different way).

So if the people who trained the model decided that instead of making it general and versatile they will only focus on higher quality content, and made a manual selection if titles (most academic books, especially from respected publishers; public source code only from prominent projects with and adequately high number of developers and so on), they could produce a model that would be less eloquent but could generate less bullshit.