This idea that because a human is sometimes wrong, it's okay for the chatbot to be wrong is one I find to be dangerous and unhelpful. The fact that people are fallible shouldn't encourage us to choose solutions with a high or unpredictable error rate. Some failures have a greater cost (i.e. launching a rocket) than others (delivering a package).

Does the chatbot misdiagnose less than a doctor? When the chatbot does misdiagnose, is it in manner similar to a doctor? In the same way that AI generated images always had weird and creepy fingers, does this chatbot consistently misdiagnose the common cold as brain cancer?

IMHO, without some real data, we're simply experimenting on people. That is wholly immoral.

After some contradictory information on error rates and failure modes, a Google employee is quoted in the article:

'Corrado said that while he wouldn’t want it to be a part of his own family’s “healthcare journey,” he believes Med-PaLM 2 “takes the places in healthcare where AI can be beneficial and expands them by 10-fold.”'

Yikes! 10-fold expansion, excluding any interaction with people this person really cares about. Let us remember that this is a 10x times increase over the current "places where chatbot are beneficial in healthcare", that is to say zero.