See also: OpenAI's Paul Christiano wrote on the same topic — https://www.lesserwrong.com/posts/AfGmsjGPXN97kNp57/argument...

(cf. https://intelligence.org/files/IEM.pdf for some of the arguments being argued against)