By any reasonable standard, a “clinical pharmacist for 10 years” should understand that there are circumstances where there is a correct answer, and incorrect answers are potentially catastrophic.
You, clearly, are either unable to recognize that or are unable to recognize that LLM are dangerously useless in such contexts. Either you already know the correct answer, making this useless, or you are not competent to utilize its answers because you can’t tell whether it’s about to kill someone with plausible nonsense.
And if a “clinical pharmacist for 10 years” can lose sight of this really fundamental issue, this whole thing needs to be halted. Governmentally if necessary.
And it won’t be.
It’s bad enough that “bad actors” will leverage this for their purposes, but the presumably competent simply turning off their brains over this is the most perplexing glitch ever to be on public display.