Hacker News Reader

Comment by randycupertino | Hacker News Reader

I had a weird LLM use instance happen at work this week, we were in a big important protocol review meeting with 35 remote people and someone asks how long IUDs begin to take effect in patients. I put it in ChatGPT for my own reference and read the answer in my head but didn't say anything (I'm ops, I just row the boat and let the docs steer the ship). Anyone this bigwig Oxford/Johns Hopkins cardiologist who we pay $600k a year pipes up in the meeting and her answer is VERBATIM reading off the ChatGPT language word for word. All she did was ask it the answer and repeat what it said! Anyway it kinda made me sad that all this big fancy doctor is doing is spitting out lazy default ChatGPT answers to guide our research :( Also everyone else in the meeting was so impressed with her, "wow Dr. so and so thank you so much for this helpful update!" etc. :-/

View on HN

The LLM may well have pulled the answer from a medical reference similar to that used by the dr. I have no idea why you think an expert in the field would use ChatGPT for a simple question, that would be negligence.

A climate scientist I follow uses Perplexity AI in some of his YouTube videos. He stated one time that he uses it for the formatting, graphs, and synopses, but knows enough about what he's asking that he knows what it's outputting is correct.

An "expert" might use ChatGPT for the brief synopsis. It beats trying to recall something learned about a completely different sub-discipline years ago.

This is the root of the problem with LLMs.

At best, the can attempt to recall sections of scraped information, which may happen to be the answer to a question. No different to searching the web except you instantly know the source and how much to trust it, if you search yourself. I've found LLMs tend to invent sources when queried (although that seems to be getting better), so it's slower than searching for information I already know exists.

If you have to be more of an expert than the LLM to then verify the output, it requires more careful attention than going back to the original source. Useful, but it's always writing in a different way to previous models/conversations and your own writing style.

LLMs can be used to suggest ideas and summarize sources, if you can verify and mediate it. They can be used for a potential sourcing of information (and the more data agreeing, the better). However, they cannot readily be used to accurately infer new information, so the best they can do here is guess. It would be useful if they could provide a confidence indicator for all scenarios.

She read it EXACTLY as written from the ChatGPT response, verbatim. If it was her own unique response there would have been some variation.

What makes you think the LLM wasn't reproducing a snippet from a medical reference?

I mean it's possible an expert in the field was using ChatGPT to answer questions but is seems rather stupid and improbable doesn't it? It'd be a good way to completely crash your career when found out.

>her answer is VERBATIM reading off the ChatGPT language word for word

How could it be verbatim the same response you got? Even if you both typed the exact same prompt, you wouldn't get the exact same answer.[0, 1]

[0] https://kagi.com/assistant/8f4cb048-3688-40f0-88b3-931286f8a...

[1] https://kagi.com/assistant/4e16664b-43d6-4b84-a256-c038b1534...

We have a work enterprise GPT account across the company.

How does that explain what you observed?

The only way I can understand that as an explanation is if your entire company can see each other's chats, and so she clicked yours and read the response you got. Is that what you're saying?

How else would she have been able to parrot the exact same GPT response without reading it directly? You think she just thought of it word for word exactly the same off the top of her head?

They're saying that the shared account is enough for OpenAI to provide the same result. Very interesting, I'd like to know more like was it a generic IUD or a specific one in the query. Also, the Doc is a cardiologist, they don't specialize in Gyno stuff and their training/schooling is enough for them to evaluate sources.

Just for reference before AI it was typical for employers of doctors to pay for a service/app called UpToDate which provided vetted info for docs like google.

There were several specific brands cited in the response and she read through them one by one in the same order with the supporting details, word for word. I think it just gave us the same response and she read it off the page.

    Itemized Bill:
    - hitting the ChatGPT button: $1
    - knowing where to hit the ChatGPT button: $600k/year

I know some GPs that use WebMD so it plays out like:

- Google search: free

- Having the schooling, training, and experience to evaluate the results: $300k per year

- Google search: free

- Having the license to make it "official": $300k per year

The one thing a cardiologist should be able to do better than a random person is verify the plausibility of a ChatGPT answer on reproductive medicine. So I guess/hope you're paying for that verification, not just the answer itself.

Or both the doctor and ChatGPT were quoting verbatim from a reputable source?

Reply on news.ycombinator.com