Health issues aren’t addressed with accurate information, they’re addressed by understanding the needs of the individual. Even if GPT-4 could guarantee accuracy when discussing self-harm, that would not necessarily be the right answer from the perspective of ensuring GPT-4 does the most amount of good.
If a friend told me that they were suicidal, I could explain to then in great detail about the nuances of depression and medication and suicidal ideation and how to effectively harm themselves if that’s what they want, but I know that is probably not the right answer, and the right answer is actually, “I’m here for you and I will help you get professional help”.
Harm reduction often involves helping people do dangerous things more safely (like safe drug injection) but that’s one component of helping people, the key to harm reduction is the long term investment in addressing the problem. Safe injection, for example, is often married with further healthcare. GPT-4 can’t do that and so telling you to go to a healthcare professional instead is going to have a much better outcome.