I agree that belief plays a role, and that it would be difficult to perfectly replicate interactions between human beings, but I think you can get partly there, and especially for those that don't have human beings that they can interact with, it could provide benefits to them.

I've been working on my own for some time now, and I find talking to an AI via text or voice helps me work through problems, and it does give me a partial feeling of having a coworker.

Personally, while I have various social connections, there are certain things that I find interesting that none of my social connections do, and so there are certain conversations I cannot have with my social group. I sense engaging with an AI about these topics could give me more pleasure than talking to a disinterested friend about it. An interested friend would still be the best case scenario, but that is why I was thinking a product like this could be useful for those that don't have friends.

As @bsenftner mentioned as well, if you've withdrawn from social situations due to past interactions, because you feel misunderstood, or aren't good at expressing yourself, then having this no-risk platform to experiment / practice socializing with a conversational AI could be something that appeals to you, and over time gets you to a place where you seek out the real thing.