Hacker News Reader

Comment by not2b | Hacker News Reader

Yes, the ads accurately described the study. Several friends of mine participated (I live in Santa Clara County).

Of course there is selection bias, which is why they make adjustments to make the sample better represent the population (see the abstract for details on this, or read the paper for all the details, there's well-established science behind these adjustments and it's up to the reviewers to be sure that they did it correctly). The point is that "they only found 50" is quite a large number compared to the number of confirmed cases.

View on HN

What adjustments were made to account for selection bias of the sort described? It's not covered in the paper, and "well-established science" certainly doesn't answer the question.

These are very common statistical methods when dealing with surveys and populations. It isn't this paper's job to describe the background on something like that, just like it isn't this papers job to explain blood testing.

They describe their limitations and adjustments exactly like most other studies of this sort do. For example:

"Those imbalances were partly addressed by weighting our sample population by zip code, race, and sex to match the county. "

They said that they collected prior clinical symptoms in the survey, but have not mentioned that they adjusted for those.

Given that, I think it's fair to say that they have not adjusted for the selection bias of people with prior clinical symptoms being more likely to click the ad.

On the contrary - detecting selection bias of this sort can be very difficult. Do you measure it using a questionnaire? You might ask your subjects "Do you think you previously had COVID-19?" But what do you do with that response data? Exclude everyone that says yes? Then you're skewing one way. There's obviously no population data to match that response with like there is for general demographics such as race and age.

Many studies are only as good as their sampling methods. A good sampling method will alleviate concerns of selection bias.

I think you'd call individuals(using a method with far less self selection bias) and get them to fill out the same questionnaire and then do the same types of adjustments you'd do with race or gender.

But I also agree they 100% did not do any of these adjustments if they did it would be listed right here

> We report the prevalence of antibodies to SARS-CoV-2 in a sample of 3,330 people, adjusting for zip code, sex, and race/ethnicity. We also adjust for test performance characteristics using 3 different estimates: (i) the test manufacturer's data, (ii) a sample of 37 positive and 30 negative controls tested at Stanford, and (iii) a combination of both.

Adjusting for selection bias is really hard and potentially first-order.

Correct. For some things you could use baseline data, but there is zero baseline data to use here.

That reweighting doesn't address the most troubling selection bias: people choosing to participate because they want to get tested (e.g., because they think they might have had the disease).

Reply on news.ycombinator.com