I can't escape it when Google's implementing their AI to call and message other people. They just demonstrated Google's AI calling and trying to converse with unsuspecting users who aren't even notified they're talking to Google.
But right now, people being called have an expectation of talking to a person, and this shifts that expectation without the business opting-in to the change.
That's going to be frustrating to some businesses, and there may well be legal issues in some areas.
The reason people don't like phone trees is not because they are robots. Rather, because they often make it difficult to get the help you need or reach the right department. If a live human read the script of a phone tree to you it would be equally (more?) infuriating.
If I called a companies number and got a conversational bot masquerading as a person instead of a phone tree, that would be a dramatic improvement.
That may be a trivial case, but more generally our social contract is that we owe a duty of politeness and consideration to fellow humans. When you're interacting with another human, a big part of your brain is actively engaged in following social norms. By masquerading as human, machines co-opt this part of your brain and extract a duty that they don't deserve.
I think every machine caller should be required to identify itself as such. Restaurants will still be happy to take reservations from them. If one calls me to sell me something, I can tell it where to go without any guilt.
Voice will make this a LOT worse, not better.