“Judgment” is close to my mental model, but I prefer “liability”. All meatspace-based employment comes down to liability. A McDonald’s shift worker is liable for any mistakes made during their shift that are considered their responsibility. A SWE is liable for any mistakes made during their employment that are considered their responsibility. Accountant. Paralegal. CEO.
AI will need to be able to experience consequences as a result of liability, and care about those consequences, in order to replace true meatspace jobs. Otherwise they’re simply sophisticated systems.
If you’re a one man company, and you have a delivery AI that delivers widgets to Alice, but in the process that delivery AI kills Bob, you’re liable for murder.