This falls safely into the Uncanny Valley:
http://news.ycombinator.com/item?id=120425
http://www.androidscience.com/theuncannyvalley/proceedings20...
http://news.ycombinator.com/item?id=120425
http://www.androidscience.com/theuncannyvalley/proceedings20...
After all, if it LOOKS human and it engages in vaguely human-like activities (though perhaps nothing too specific, lest it slip up and show itself to be obviously virtual), I'm sure it could convince a large number of people that it is. Suddenly we have a face for that chat bot that made the news a few months ago for talking lonely young men out of their personal information . . .
Hmmm, . . ., is "rook lonely men out of credit card info" a good business plan for a YC app? :-P
The Eliza Effect meets LonelyGirl15. - Well put.