People are used to interact with other entities that can perform advanced logical inferences, deductions, etc in an intuitive way. They are at a loss when they're forced to dumb it down to "talk" to someone who needs to be told not to crash into the wall when being told to walk in a certain direction.
Worse still, when people do understand that computers are really dumb, but actually good at following orders when you treat them like a mentally challenged toddler, they get stuck again when even that fails; when they encounter application after application that does not operate in any logical sense, because they aren't programmed by other computers, but by lazy/dumb/stressed/ignorant programmers like me.
The best quote I read on this comes from this essay: http://stilldrinking.org/programming-sucks
"The only reason coders' computers work better than non-coders' computers is coders know computers are schizophrenic little children with auto-immune diseases and we don't beat them when they're bad."