Software is just language, translating human intent into concrete machine action. That translation being automated doesn't mean, the intent had been made superfluous.

When no human knows what the machine is doing exactly, how do you know your intent is being followed? The seeming sufficiency of superficiality the public "enjoys" is an illusion needing upkeep.

Machines cannot make meaning, nor can they originate intent. They can't keep themselves suspended mid-air, detached from human understanding. You need engineers who understand human and machine, identifying the reasonable among the un-reflected wishes and making it explicit and precise.