Hacker News Reader

Comment by compiler-guy | Hacker News Reader

These are very common statistical methods when dealing with surveys and populations. It isn't this paper's job to describe the background on something like that, just like it isn't this papers job to explain blood testing.

They describe their limitations and adjustments exactly like most other studies of this sort do. For example:

"Those imbalances were partly addressed by weighting our sample population by zip code, race, and sex to match the county. "

View on HN

On the contrary - detecting selection bias of this sort can be very difficult. Do you measure it using a questionnaire? You might ask your subjects "Do you think you previously had COVID-19?" But what do you do with that response data? Exclude everyone that says yes? Then you're skewing one way. There's obviously no population data to match that response with like there is for general demographics such as race and age.

Many studies are only as good as their sampling methods. A good sampling method will alleviate concerns of selection bias.

I think you'd call individuals(using a method with far less self selection bias) and get them to fill out the same questionnaire and then do the same types of adjustments you'd do with race or gender.

But I also agree they 100% did not do any of these adjustments if they did it would be listed right here

> We report the prevalence of antibodies to SARS-CoV-2 in a sample of 3,330 people, adjusting for zip code, sex, and race/ethnicity. We also adjust for test performance characteristics using 3 different estimates: (i) the test manufacturer's data, (ii) a sample of 37 positive and 30 negative controls tested at Stanford, and (iii) a combination of both.

They said that they collected prior clinical symptoms in the survey, but have not mentioned that they adjusted for those.

Given that, I think it's fair to say that they have not adjusted for the selection bias of people with prior clinical symptoms being more likely to click the ad.

Adjusting for selection bias is really hard and potentially first-order.

Correct. For some things you could use baseline data, but there is zero baseline data to use here.

That reweighting doesn't address the most troubling selection bias: people choosing to participate because they want to get tested (e.g., because they think they might have had the disease).

Reply on news.ycombinator.com