Claude and LLaMa get basic human anatomy wrong | Hacker News Reader

Claude and LLaMa get basic human anatomy wrong

hermitian.substack.com

12 pointsjohntiger129 comments

As a cardiologist I will translate this into software: saying that the name of the portion of the diaphgragm that opens to the IVC is "basic human anatomy" is like saying that "monoids in the category of endofunctors" is a basic tenet of software development.

Don't think I learned about monoids in 1st year CS. I have learned "I ate 10 eggs at 12" in 1st year medical school.

Your first year of medical school is after your 4 years of undergrad.

Not in Quebec or the UK.

And do you think that “caval hiatus,” the phrase that you used in your prompts, is the term most likely to elicit that from LLMs?

I ask because using plain medical language seems to elicit the correct answer from Claude and ChatGPT3.5 (I didn’t test llama from my phone.)

As a radiologist who looks at thoracoabdominal anatomy for a living I completely agree. That was also my initial reaction, I don't think I've ever read "caval hiatus" used in clinical text other than maybe during medical school.

It's not even correct, anatomically it's most correctly described as the caval foramen. Hiatus refers to a slit/gap, foramen refers to a circumscribed orifice. Caval hiatus is either misremembering or an archaic text. Clinically people use aperture, foramen or opening.

Other problems with this post:

I assume the incredibly poor way of prompting is attempting to simulate a user using a chat interface? If so, are they even using the right LLaMA model (at the very least chat fine-tuned, ideally also on the open medical conversation dataset)? What GPT model are we comparing with?

Is there system context provided for the prompt? If you were deploying this you wouldn't only pass the inputed user text in the prompt. ChatGPT's interface adds this so if you're going to make a fair comparison you should be doing the same.

It's still going to favor OpenAI for this specific use case (vague and poorly phrased short prompts of esoteric trivia) as they've had much more training in this domain.

As a non-native speaker I wonder if this could still be of use because it's maybe better than using a dictionary. I think getting medical terms right is one of the hardest things to do in a hurry (like when visiting a doctor on vacation, not if you're living in an English-speaking country). You're only guessing anyway.

Can't believe I'm somehow defending bad LLM output, but here I am, even with this huge caveat,

Potentially, but no patient or physician is going to be using "caval hiatus".

If this was the intention, the author should disclose that he's intentionally using very obscure antiquated wording not "basic human anatomy". If he is instead unaware that "caval hiatus" is not accepted terminology it is further evidence supporting how obscure this fact is.

It's literally being taught in classes at a major Canadian medical school, in 2023.

Which means nothing, you have a pre-clerkship anatomy course director using archaic and obscure terminology that is extremely rarely used (and arguably incorrect, but we don't have to debate etymology). You will never see this used clinically or in any major anatomy textbook.

As you are a medical student you should be familiar with accepted references for anatomy. In case you're not here are some:

Netter's anatomy (most commonly used clinical anatomy text): caval foramen.

Gray's anatomy: inferior vena cava opening.

eAnatomy: caval foramen.

Your other clue is in ChatGPT's response:

>The term "caval hiatus" is typically used in reference to human anatomy, specifically the anatomical structure known as the "caval opening" or "caval foramen."....

If you ask Claude the same question using 'caval foramen':

>The caval foramen, also known as the foramen of vena cava, is located at the level of T8 vertebra in the thoracic region of the vertebral column. Specifically...

Some friendly advice as you seem to be early in your medical training:

1. When an attending (in this case two) corrects you, you're probably wrong. Your next step should be to confirm your knowledge not retort with "it's what I was taught".

2. Never say a "major medical school" in Canada (where I also graduated from) which implies there are tiers. Unlike the US, all Canadian medical schools and (almost all) residency programs are excellent.

I disagree with the idea that this is basic human anatomy, basic at the level of a practitioner, likely, basic at the level of a lay person like myself? Not certain. Had you asked me "Hey Ian, at what level is the caval hiatus?" I would either tell you I had no clue or look it up.

If anything this demonstrates that LLMs aren't great with niche knowledge, which makes sense since the training corpuses are constructed from common knowledge for the most part.

This is however, a potentially great case for retrieval augmented generation.

Interestingly, we actually do exactly that, although our underlying LLM is OpenAI-based, so we would have avoided the error in the first place :)

> I disagree with the idea that this is basic human anatomy

When I say basic, I mean something taught in second or third year anatomy or physiology courses; or first year medical school.

Fair enough, It's still more specialized knowledge and I'd guess Anthropic's training corpus is light on it. If I recall correctly OpenAI included a lot of textbook knowledge.

No you're right. He's using an incredibly obscure term (caval hiatus rather than caval opening/foramen/aperture) for a mild-moderately obscure question.

Nevertheless I would wager most clinicians (except for surgeons, radiologists or cardiologists) wouldn't know the answer to this question. You learn this in first year of medical school and then forget it.

Out of curiosity what stage are you in your medical training? You will never hear "caval hiatus" after pre-clerkship.

I mean, it's a well known fact that LLMs do badly in the "long tail" of the distribution: anything they haven't seen many many times (e.g. relatively obscure specialist knowledge), they are definitely very likely to get wrong. The more "popular" the thing you're asking from them is, the more likely it is they'll answer correctly.

I don't think it's that obscure. It's something taught in first-year medical school, or in undergrad anatomy or physiology. Hardly anything that deep

Really? It's not even the most common way to refer to the caval foramen. This is obscure archaic terminology.

It's caval foramen, esophageal hiatus and aortic hiatus. Clinically, everyone says opening, foramen or aperture.

FWIW, Pi returns: "The caval hiatus is located at the level of the T8 vertebra, which is near the middle of the spine. It's a small opening in the diaphragm that allows the inferior vena cava, a major vein, to pass through. There's also a nearby opening called the esophageal hiatus, which is where the esophagus passes through the diaphragm. The caval hiatus is an important anatomical landmark, and it's also sometimes called the caval opening or the caval orifice. Is this something you're studying for a class or just out of curiosity?"

Interesting, haven't heard of Pi. Is this a custom LLM?

ChatGPT uses a 1.7 trillion parameter model IIRC. I wouldn't expect a 70b model to compare well. There are medical finetunes that might be better to try for this, like https://huggingface.co/wanglab/ClinicalCamel-70B

Edit/Correction: as mentioned below, free tier used by OP is in the neighborhood of 175b

Where are you getting the 1.7T number from? Genuinely curious, as sources like [1] generally cite 175B.

[1]: https://ecoagi.ai/articles/chatgpt-parameters

I'm not sure, I've tried looking it up and couldn't find an official answer. But that site says GPT3 is 175b and GPT4 is 170 trillion. That's insane if true

Edit: and this one says 1.7, idk https://levelup.gitconnected.com/gpt-4-parameters-explained-...

Haven't used chatgpt in a bit, but looks like the free tier is GPT3 still, so 175b may be correct if OP isn't a subscriber

I was testing on GPT3.5-Turbo, the free tier.

Ah, 175b it is then. That's actually kind of encouraging that an open source trillion+ param model isn't needed to get there.

A 180b Falcon model very recently came out https://huggingface.co/tiiuae/falcon-180B A finetune of this might be competitive, but I haven't tried it.

That should be in your article.

I wonder if it would do better with a better prompt?

Why Claude and LLaMa get it wrong but OpenAI's LLM get it right?

Perhaps more time with RLHF due to first mover advantage? If you figure it out, Meta and Anthropic would probably pay big $$ for the secret

Reply on news.ycombinator.com