It's pretty easy for a reasonable person to determine when to call 911 and when to ask an LLM. Typically when I ask an LLM, it is for something I am not willing to escalate further at the moment.

Anthropic has a significant interest in reducing liability, which in this case means mostly blocking this use case.

Have you heard the cost of health insurance and seeing a doctor? Some people don't see going to hospital as an option. Better for them to have a local model than no options. But let me tell you how I use it for health, I take all my health records, lab results, every thing, I throw it in there for a second opinion, it explains things to me, it suggests questions I should ask on follow up, possible options, etc.