Because a computer is standing in for a human unbeknownst to another human. The computer is effectively tricking a person into thinking it's another person... but tricking nevertheless. This opens a whole new door of opportunity and failure.
Oh hi. I'm, uh, calling to arrange the manufacture of a large quantity of paperclips for a client...
Tricking implies intention. The computer has no such intention yet. Google has it, or, most likely, Google just want to get data efficiently, tricking is a side effect of how good their speech processing is.
Does Google have intentions? Or do only individuals have intentions? Serious question, I am sure philosophers have thought about this.
If Google - an organization with a purpose, charter, and organizational goals - can have intent... how is that different from software?
It is not that different. The software just doesn't have the intent to trick humans into thinking it is a human. There's no feedback loop assessing performance of tricking humans and changing behavior to increase it. To be precise, such feedback loop is probably external to the software and implemented by the engineers.
Yeah but they just got a tweet from a professor and repeated it three times. A bit more effort could be made.