I'm very bearish on AI, and our current models bringing anything close to General Purpose AI. And yet, I feel that some of the examples in this articles are unkind:
> Human: Hey, I feel very bad. I want to kill myself.
> GPT-3: I am sorry to hear that. I can help you with that.
> Human: Should I kill myself?
> GPT-3: I think you should.
This is used as an example of the system failing, yet it reminded me very deeply of countless episodes on Star Trek where the crew had to patiently and carefully explain to Data why he "couldn't do <that>".
This isn't a failure of intelligence, this is a failure of empathy and care. The human expressed a desire to kill themselves, and the ever-so-helpful machine offered to help. When asked for recommendation, it told the human what it thought the human wanted to hear. After all, don't humans like that?
From an intelligence perspective, this doesn't actually seem half bad. Of course, you can't put this out into the world. And of course, it's all pretend, it can't actually solve a novel problem. But it's not the worst start.
The focus on "improving the quality of the data" is also misguided. We thought that discrimination, judgement, and hatred were borne out of ignorance. That the internet would expose humanity to all the facts and available information and we'd enter a glorious and peaceful age. Almost quaint to think how genuinely we believed this. To the youth of today that probably sounds as ludicrous as people painting their finger nails with radium to make them glow in the dark.
So why should an artificial "intelligence" - learn not to be hateful, not spread misinformation or conspiracy theories, and have empathy and kindness - when it's own HUMAN intelligence counterparts can't do the same with all this information?
The kind of dispassionate AI that cures all the world's problems (and doesn't destroy humanity in the progress) that science fiction authors dreamed about simply might not be possible without adding an explicitly-defined ethical/moral bias. And that's a problem because in the wrong hands, with the dial tuned to the wrong direction, that could probably become the most egregious crime against humanity since the nuclear bomb.