> In my opinion being 100,000x lower cost gives such a system some leeway in being worse and I think most patients would agree.
I would heavily disagree with this. We had an AI counselor actively encouraging disordered eating and triggering people into relapse on an eating disorder helpline from logic like this.