Hacker News Reader

Comment by nicklecompte | Hacker News Reader

I haven't finished reading the paper closely, but it is disturbing that this is the only real reference to accuracy and relevance I could find:

> Barriers to adoption include draft message voice and/or tone, content relevance (8 positive, 1 neutral, and 9 negative), and accuracy (4 positive and 5 negative).

That's it! Outside of that it looks like almost all the metrics are about how this makes life easier for doctors. Maybe there is more in the supplement, I have not read closely.

Look: I am not a policymaker or a physician. It's quite plausible that a few LLM hallucinations is an acceptable price to mitigate the serious threat of physician overwork and sleep deprivation. And GPT-4 is pretty good these days, so maybe the accuracy issues were minor and had little impact on patient care.

What's not acceptable is motivated reasoning about productivity leading people to treat the accuracy and reliability of software as an afterthought.

View on HN

Those are just free-text comments from the ~80 physicians who used this tool. Tables 2-4 show the researchers measured various characteristics like message read/write time, utilization rate, work load, burnout, utility, and message quality.

It's also worth noting that, from the abstract, the study's objective isn't to study the LLM's accuracy. This is a study of the effectiveness of this drafting system's implementation in the hospital. I'm not saying the accuracy isn't an important component of the system's effectiveness, but it's not the question they're answering.

My point is that "accuracy is not the question they're answering" is a fatal flaw that makes this research pointless.

Say I released a linear algebra library and extolled its performance benefits + ease of development, then offhandedly mentioned that "it has some concerns with accuracy" without giving further details. You wouldn't say "ah, he focused on performance rather than accuracy." You'd say "is this a scam?" It is never acceptable for healthcare software researchers and practitioners to ignore accuracy. LLMs don't change that.

The only thing that sort of cynical laziness is good for is justifying bad decision-making by doctors and hospital administrators.

It is worth studying the quality of the generated messages, but the system's effectiveness does not hinge solely on the quality of those messages.

There's not a singular metric like accuracy that summarizes the quality of a generative system. What would an "accurate" message be? I can think of several aspects: mimicking the doctor's tone, addressing the correct patient concern, not providing clinically false information, not being too long, using the correct language. A good statistical characterization of this system requires rating multiple aspects of the content. These researchers measure the system's quality through user surveys and interviews. Is there a better ground truth to compare it to?

There's an important distinction between accuracy and effectiveness. These researchers are interested in the system's effectiveness. An algebra library can't be effective without being 99%+ accurate. Plenty of tools can still be useful without being perfectly accurate. Diagnostic tests are a great example. Conversely, you can have tests that are 99% accurate, but not useful, depending on the context and implementation.

I think that's why these researchers are less concerned with accuracy. Effectiveness is a separate concern from quality/accuracy and can be studied separately. They're different research questions.

Reply on news.ycombinator.com