Sounds like @benl has been afflicted with the "Cartesian wound". Such dualistic thinking and ideas like free will are ~hard for us to work through. But perhaps the more important, and immediately tractable, question @benl brings up is what our approach should be? Should we make an AGI or better IA, Intelligence Augmentation?

~hard

Daniel Dennett

From Bacteria to Bach and Back