Hacker News Reader

Comment by something98 | Hacker News Reader

This is a very misleading statement; most of those physicians are using LLMs to transcribe notes from visits and/or for billing purposes (e.g., proper billing codes).

View on HN

The problems isnt LLMs per se, it is the shift to trusting the output of the machine coupled with a decline in verifying that the output is reasonable. It's basically what your teachers warned you about with wikipedia in eight grade except applied to all areas of life, including medicine. Dictation is already high-stakes and LLMs do not automatically reduce that risk.

Here is an example. My provider sent me this note. I'm quoting verbatim here from my MyChart record:

"Your liver enzymes are high, I would like to order acetaminophen containing medication like Tylenol, I would like to order liver ultrasound I placed ultrasound order in the system, make an appointment for radiology, I would like you to get hepatitis panel lab work done, obtain blood work order, please schedule a well visit to get it done"

When I queried it, this is what I got back. It was a dictation error. You could almost hear the panic in the message:

"Sorry for wrong message earlier, I was dictated message- so could not realize that it was written to take Tylenol type of medicines- I DO NOT RECOMMEND ACETAMINOPHEN CONTAINING MEDICINE - LIKE TYLENOL AND ALCOHOL DUE TO ELEVATED LIVER ENZYMES."

Again the problem is not dictation, or LLMs. The problem is humans ignoring their responsibility to check the output of a machine.

> Again the problem is not dictation, or LLMs. The problem is humans ignoring their responsibility to check the output of a machine.

100%. Also, management.

I wish someone would go ahead and coin an AI version of Amdahl's law that states the work speedup from AI is dependent on amount of unverified AI output used.

Iow, if you 1:1 verified everything, there would be no time savings.

Ergo, you get management saying (1) we demand time savings due to AI & (2) we demand you fully check anything you use AI for.

End result? People skip (2) to hit (1).

Then management burns anyone at the stake whenever inevitable mistakes happen.

But that’s trivially false. There is an entire category of work where it is hard to come up with an answer and easy to verify the answer, which means that if you verified everything there would still be a large time savings.

I would question whether that holds in the practical LLM automation space.

Can you think of any real life examples where an LLM is likely to be used?

I think in practice what you're saying is there are problems where there exist efficient deterministic verification methods, and I'm sure that's true.

But that's not the bulk of everyday work LLMs are being asked to do nowadays across industry.

You're questioning whether there are any interesting problems in the P v NP space? https://en.wikipedia.org/wiki/P_versus_NP_problem

But if you want to keep it in the realm of the everyday: you're asking if it is easier to write an email than to read it and check it covers what you wanted to say? Is it easier to search for something or to look at what's been found and say that it's what you were looking for?

Which is itself a problem as (in my partners evaluations as an optometrist), LLMs used for clinical notes has a bad habit of dropping clinically important information, and the biggest providers don’t give you a copy of the raw transcript or a recording

Which means she ends up spending just as much time as if she’d done it herself as it needs to be verified for accuracy every time…

OpenEvidence is specifically meant to help clinicians make evidence-based decisions in the diagnosis and treatment of patients, not note transcription.

It does both: https://www.openevidence.com/user-guide/visits-overview

Reply on news.ycombinator.com