Hacker News Reader

I think the parent is saying what if the AI made such a terrible mess that the team of imperfect people thought it was fine, but it was just as bad as the terrible mess the team would have created because the team is not capable of evaluating whether it's a good idea or not. (possible follow on consequences -- no one can debug it or figure out if it's a good idea either)

As the two other commenters pointed out

The mess already existed for a reason. There’s a certain amount of expertise in the average firm.

If they could afford an expert, they wouldn’t be the same firm.

If they do get an AI expert - how do they check the output given the level of ability they have?

The point is that in real companies the bad mess already exists. So it is a good argument. Or at least a practical one.