I’m getting flashbacks to reading Asimov’s Robot series of novels!
1. A robot may not injure a human being or, through inaction, allow a human being to come to harm.
… etc…
1. A robot may not injure a human being or, through inaction, allow a human being to come to harm.
… etc…
You need something like a calculus of morality and ethics - this is incredibly uncomfortable for people, because it will mean the invalidation of moral relativity and all sorts of arbitrary dogmatic and ideological tradition, and demonstrate a rational basis for intersubjective interaction. ( Take your is/ought distinction and bury it with Hume.)
We need progress, and the sooner we start, the less damage will be done by unaligned systems.