Hacker News Reader

I'm a PhD in clinical psychology, currently working with clients, whose research has focused on the psychometric evaluation of questionnaire measures.

I have no idea what you're talking about. The diagnostic criteria are not "very broad because they are necessary as tools". The diagnostic criteria are meant to be sufficient criteria for diagnosing a disorder. Questionnaire-based measures act as a scientifically validated proxy for a clinical interview to determine this. With a sufficient score on one of several validated diagnostic questionnaires, you can typically say with 80-90% certainty (specificity is the technical term for that percentage) that the person would receive a diagnosis from a clinical interview.

Psychologists and epidemiologists can and do research prevalence of mental health conditions by randomly sampling the population. This does not "over-diagnose" mental health disorders. That's absolute nonsense. Over-diagnosis is only an issue in cases where prevalence is below the false positive rate (technical term is "sensitivity") of the test. That is not the case here.

If you looked at the linked CDC study, they even mentioned the instruments that were used to support this statement. The PHQ2 and GAD2 are validated very brief measures of depression and anxiety that are sufficient for estimating the likelihood of a depression or anxiety diagnosis respectively. Likelihood of a false positive on the PHQ2[1] and GAD2[2] are both around 20% (100-specificity), which is just fine. Both of these are more than sufficient for epidemiological purposes if not for individual Dx.

The fact of the matter is that nationwide prevalence for diagnosable disorders IS high, and it's on the rise. Coronavirus has made people anxious and quarantine has made them depressed. And a LOT of people need help.

[1] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2906530/ [2] https://www.hiv.uw.edu/page/mental-health-screening/gad-2

Is feeling anxious in a potentially dangerous situation or depressed in a depressing situation really a disorder that needs to be treated? It seems like a normal response and what needs to change is the situation, not people's reaction to it.

Isn't this like asking "is it weird to feel ill during an illness"? What I mean - shouldn't humans be able to cope with these situations as "sad", but not "depressing", as depression reduces their ability to move forward?

Anxiety and sadness can both be adaptive, yes. How we cope with those emotions can also be adaptive or maladaptive.

Both anxiety and depression, even in circumstances where they are reasonable responses, can benefit from supportive counseling, acceptance, self-compassion, etc. It's important to note that a therapist can "treat a problem" and "support and facilitate". In the second sense, all suffering can be "treated", even if it is not considered pathological.

> Over-diagnosis is only an issue in cases where prevalence is below the false positive rate (technical term is "sensitivity") of the test. That is not the case here.

The term "overdiagnosis" is actually ambiguous. As [1] points out, the word is used in two different ways: (a) to argue that the diagnostic criteria (such as in the DSM-5) are overly broad, and are labelling people as having a "disorder" who shouldn't be so labelled ("pathologising of normal human experience"); (b) to argue that the diagnostic criteria are being systematically over-applied, where people who don't actually meet the formal diagnostic criteria are nonetheless frequently being given the diagnosis.

[1] https://aifs.gov.au/cfca/publications/diagnosis-child-mental...

This is a fair point. I read the op's comment as using the former definition. The latter definition wouldn't really apply as this is being determined by standardized testing. Unless one thinks that the standardized tests are overly liberal (their criteria are based on correlations with in-person diagnoses, so it's possible).

Well, it is a legitimate question of how well the standardised tests actually fit the diagnostic criteria for various disorders. [1] argues that the fit between the commonly used standardised tests for ASD and the DSM-5 diagnostic criteria is less than ideal. (And I doubt this finding is unique to ASD, I think it likely if you repeated the authors' exercise for other diagnoses you would get similar results.)

[1] https://link.springer.com/article/10.1007%2Fs00787-020-01481...

20% false positives is not "just fine". That is a huge error.

I agree with you, but in psychiatry and clinical psychology, a 20% false positive rate is considered "acceptable".

Consider for example the Cochrane review of autism diagnostic instruments in preschoolers [1]. It found that on average (in research contexts), the main diagnostic test used for ASD, the ADOS[2], had a specificity of 0.80 (in other words, a 20% false positive rate). The Cochrane authors considered that acceptable. Now, professionals may consider that acceptable – but how many parents are told that the test used to diagnose their child has a 20% false positive rate? Added to that, there is evidence that the tests are actually less accurate in non-research contexts (the standards for administering them are certainly lower), which may make the false positive rate in non-research contexts even higher (on that topic see [3]).

[1] https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD...

[2] actually the original ADOS has now been superseded by a newer version, the ADOS-2, but the Cochrane reviewers couldn't find any studies of the ADOS-2 that met their inclusion criteria. In any case, I doubt the accuracy of the new version is substantially different from the original one.

[3] https://link.springer.com/article/10.1007/s00787-018-1143-y

These are screening tools. They are not meant to make definitive diagnoses for the most part. They are used for epidemiology and flagging clients for further assessment.

There's always a balance to be made between sensitivity (picking up on real positives) and specificity (not producing false positives). The balance for these instruments is at about 80-85%. If a person is depressed, there's an 80-85% chance they will be correctly classified. If a person is not depressed, there is an 80-85% chance they will be correct classified.

Raise the threshold and you'll be more specific but less sensitive. Lower the threshold and you'll be less specific but more sensitive. It's just how the game works and why questionnaires are only first-pass tools.

Hold on, do I understand correctly your comment about the 20% false positive rate? So even if exactly ZERO Americans were experiencing depression, a random sampling would indicate 20% of Americans are showing signs of depression? And if it's 20% for both tests, that would would correspond to a 36% false positive rate for at least one condition [0]. So the headline that a third of Americans are showing signs of anxiety or depression is consistent with what you'd expect to observe if there was zero (or very small) true prevalence? How does that work out?

[0] assuming test results for anxiety and depression are uncorrelated, which seems unlikely. But even if they were 100% correlated, the joint FPR would still be 20%.

https://jamanetwork.com/journals/jamainternalmedicine/fullar...

https://blogs.cornell.edu/info2040/2014/11/12/doctors-dont-k...

The situation is a little more complicated as these tests ALSO have a false negative rate. And that false negative rate is also about 20%. As well, the false positive rate is for scores at threshold, which is 3 out of 5. If they score a 4 or a 5, the false positive rate drops dramatically.

As well, this isn't an antibody test. Diagnosis is not dichotomous but dimensional. A false positive on depression inventory does not mean the person actually has zero symptoms. It means that their symptoms might actually be mild rather than moderate because of the way they interpreted one or more of the questionnaire items.

In general, the way to look at this is that of the people who took the test, 80% of those who scored at threshold would likely receive a diagnosis from a clinician.

This doesn't address in any way the point being made about false positives leading to overly high estimates of prevalence in the general population, whereas nullc's comment is probably quite close to explaining the issue.

> This doesn't address in any way the point being made about false positives leading to overly high estimates of prevalence in the general population

It does if you are aware of the fact that these tests are never used on its own to diagnose depression/anxiety.

The GP or psychologist determines if the client exhibits symptoms of depression, and THEN performs the test, often in combination with other tests that test similar things. Afterwards the results of those tests (as well as the dialogue that's been had with the client), are used in conjunction to determine if a diagnosis is appropriate.

I think it's rather arrogant to assume that psychologists/diagnosticians/statisticians are oblivious to the fact that tests have reliability/validity measures. Of course they are aware, that's their job.

I'm not sure what you're replying to, I made that exact point upthread and was responding to the disagreement that followed.

>"very broad because they are necessary as tools" >"diagnostic criteria are meant to be sufficient criteria for diagnosing a disorder"

Aren't these equivalent ideas? I don't think the parent comment was meant to be critical so much as it was to make an assessment. An overall rate of 80% accurate for a questionnaire instrument seems like a fine tool.

I read the comment as implying that they were overly broad and so should not be applied to the larger population outside of a professionals supervision.

My counter-argument was that they were properly narrow and could indeed be applied to the general population, though this would not provide definitive diagnoses only prevalence information.

I read the OP as saying that these statistics were not representative of the actual underlying mental health phenomena. I believe and asserted that they are.

The article shows the rise in people that meet these criteria over time, based on the same criteria self-reported in the same way.

Aren't somebody the criteria overkill in extenuating circumstances? Usually they will have caveats like "not immediately after death of a family member", etc.

This is an important point about the role of priors — especially if one is using a statistical (correlation based) proxy for what is actually a causal question.

But here’s a practical question: imagine a universal healthcare mandate which allows people to get a mental health check-up every year. What criteria should the doctors use? Would the diagnostics have to change significantly?

What do practitioners do in countries with universal healthcare?

You still need to actively persue the healthcare even if your country pays for it.

I will say the barriers are definitely lower though when i compare my home country Australia to my adopted country the USA. Sure you need to actively pursue healthcare in Australia but the barriers are just so much lower.

A great example is http://www.cancerscreening.gov.au/internet/screening/publish... You fill in the details and you're mailed a reply post paid bowel cancer screening kit.

Oh and more relevant, the portal for mental health care. https://headtohealth.gov.au/

I suspect the US government won't have anything quite like this since the healthcare is not centralized.

In Canada shrinks aren’t covered by our public health insurance. They are just as expensive as they are in the states.

If you really think you need it, you go privately or join a long (years) waiting list for mental health service, maybe get a group therapy session if you're lucky while you wait.

UK mental health service doesn't work currently.

> This is approximately the same phenomenon that causes medical students to self-diagnose themselves with every disease they learn about

Could you say more about this? It seems pretty counter-intuitive that medical students could look at some diagnostic criteria—which I assume often contain some fairly straightforward things like, "green spots on armpits" or whatever—and come out convinced they have the disease.

When you say diagnostic criteria function as a "tool," what is the purpose of the tool? My only guess so far is that it's about leaving things up to the doctor's case-by-case judgement by leaving the criteria overly broad. Maybe this ties into legal concerns as well as the fuzzy nature of real instances of diseases (i.e. it's rare to be able to exactly specify the symptoms in a way that works for all cases)?

I think the idea is that an implied symptom in diagnostic criteria is that the person was distressed enough to seek treatment. That bit of info is important for determining likelihood of having the disease.

Say you have a test for someone having a heart attack that is 99.9% accurate. If someone comes in complaining of chest pains, and the test says they are having a heart attack, then it is pretty certain they are having a heart attack.

Now, if you gave that same test to everyone in the United States, and you see that you have been diagnosed with a heart attack, most likely you AREN'T having a heart attack.

If you give this test to 300 million people, and 1 in 1000 tests give the wrong result, you are going to get a lot of false results.... 300,000 or so, in fact.

Given that the vast majority of people are not currently having a heart attack, the vast majority of those false results will be false positives.

This is why knowing the prior base rates is so important for diagnostic tests.

Hmm. So that looks like two separate issues to me, but maybe I'm missing something. I see:

1. Additional always-present criterion of distress

2. Knowledge of prior base rates as a means of estimating a proportion of false positives.

The first one makes sense to me in connection with the earlier comment. We aren't great at definitively reading our own states: it's easy for us to imagine symptoms, especially when prompted. But if something is actually wrong, an element of distress is typically present.

The second one seems like a separate issues though, or at least I don't see the connection. It seems like there are two important bits of information for correcting false positives. The first is that you have a known accuracy which suggests a 1 in 1000 chance of any diagnosis being incorrect; the second is that through historical record you can estimate how many people in e.g. the US are having a hard attack at any given moment on average.

I don't see the connection between either of those pieces of information and loose diagnostic criteria though.

The base rate part is just explaining why the diagnostic doesn't work when applied on a healthy population (like med students)

The instruction would be something like "if the patient complains of x, check for symptom y... if the have symptom y, they are likely suffering from z"

But the med student then checks themselves for symptom y, and thinks they have z, but that is only the case if they are suffering from x. The suffering from x part is what changes the base rate, and can't be ignored.

But in the patient's complaint about X, wouldn't X just be another symptom?

For instance, let's say a patient complains of a headache. The doctor uses whatever other knowledge they have of the patient to come up with some candidate diagnoses, and proceeds checking for symptoms from each of the candidates.

Presumably 'headache' is just another symptom on the diagnostic criteria for each of the other candidates. So why is it treated specially?

(Btw, thanks for your response—not sure why you were downvoted, but that was not from me!)

Right, my point is that the symptom X is often an implied one.... it isn't expressed in the diagnostic test explicitly, because the diagnostic test is run only when someone has symptom x.

> very broad because they are necessary as tools for practicing doctors seeing patients

This is such nonsense. These criteria are supposed to be scientifically valid and not a tool. What other medical speciality calls their diagnostic criteria tools. That's like saying the standards for diagnosing cancer are just a tool and anyone could be diagnosed with cancer if the doctor thought you would benefit from chemo drugs.

Amazing the mental gymnastics people will go to defend an obviously corrupt discipline like psychiatry.

In a cool future universe where we can introspect the brain like we do with code the need for statistics based diagnostic criteria might disappear but it's what we've got.

Meeting the criteria is a good sign that you should talk to someone about it and figure out if that's the best explanation for what's happening. It's not a checklist and then "you've got the GAD."

> In a cool future universe where we can introspect the brain like we do...

We can do that. It's called fMRI.

> It's not a checklist and then "you've got the GAD."

Yes it is. That's literally exactly what is it.

> We can do that. It's called fMRI.

It's clearly not sufficient or not ready yet otherwise it would be used as such.

> Yes it is. That's literally exactly what is it.

That's not a fair assessment.