6. Louise-Anne McNutt and Ketcham point to a possible detection error in the study. Namely, "...according to Maricopa County guidelines, students are considered 'close contacts' of an infected student—and thus subject to potential testing and quarantine—only if they (or that infected student) were unmasked. As a result, students in Maricopa schools with mask mandates may have been less likely than students in schools without mandates to get tested following an initial exposure." To this the study authors responded that it is, "highly speculative to make the assumption that identified close contacts are more likely to be tested than other students." This is a little harder to muddle through as we are getting a core argument on designing a reliable study. So, we are looking at the definition of an outbreak. The study states its definition thusly, "A school-associated outbreak was defined as the occurrence of two or more laboratory-confirmed COVID-19 cases§ among students or staff members at the school within a 14-day period and at least 7 calendar days after school started, and that was otherwise consistent with the Council for State and Territorial Epidemiologists 2020 outbreak definition¶ and Arizona’s school-associated outbreak definition." The CSTE defines an outbreak thusly, "Two or more laboratory-confirmed COVID-19 cases among students or staff with onsets within a 14-day period, who are epidemiologically linked, do not share a household, and were not identified as close contacts of each other in another setting during standard case investigation or contact tracing." The confounding factor that McNutt and Ketcham are referring to, which is that if two people are considered an outbreak they could have caught covid from outside of the school instead of each other, is sort of moot here as the paper states they conducted adjusted logical regression analysis against many factors including the, "7-day COVID-19 case rate in the school’s zip code during the week school commenced". That means that detection bias should not be present and we can be fairly certain that the definition of outbreak in this study is both consistent and likely means that the two or more cases are in fact school related.
7. Jason Abaluck makes the point that vaccination status could be a confounding variable. It can be! This is a flaw in the study that the study itself acknowledges. I'm not sure how this is an argument for the study being junk science. It's a little like one of the first probability and statistics examples I was taught. In this example a study was done on Coney Island where the scientists looked at ice cream sales and the number of drownings in the area. They found that there is a clear correlation between them as when ice cream sales went up so did drownings. In the study they unfortunately could not get access to temperature data or visitor counts. They concluded that ice cream sales are a good indicator of drownings. Of course, the reason ice cream sales and drownings are correlated is because people go to Coney Island when it is hot out. They also eat ice cream when it is hot out. With more people going to Coney Island there are more people that are likely to drown. The study did not make the claim that ice cream sales cause drownings, just like this study does not claim that mask mandates reduce covid outbreaks. However, in both cases the indicator is useful. In the case of mask mandates we also have data that shows masks reduce covid transmission in many other scenarios. So, yes, vaccination status is a known confounding variable that I'm sure the authors of the study would like to control for. Since they can't they did the next best thing and established that mask mandates are a good indicator for reduced covid outbreaks.
8. David Zweig attempts to reproduce some of the data himself. He attempts to build part of the data set used in the study for Maricopa county. In the end he gets the list of schools from the study authors. He writes, "Yet it still included at least three schools in Pima County, along with at least one virtual academy, one preschool, and more than 80 entries for vocational programs that are not actual schools. In response to a follow-up inquiry, they acknowledged having included the online school by mistake, while attributing any other potential misclassifications to the Arizona Department of Education." This is interesting, but ultimately the misclassification of some schools does not ultimately change the statistical methods used.
This is it. Eight arguments meant to show, "...the study’s methodology and data set appear to have significant flaws." I don't see the points made as exposing significant flaws. However, I am a simple programmer who studied mathematics and reads technical specifications in my spare time.
At the end of the day please remember these truths. Covid kills a lot of people, mainly the older ones. Covid transmits via aerosolized bodily fluids (coughs, sneezes, breathing), and things like wiping your nose then touching doorknobs where someone does the same in reverse order. Masks, preferably well fitting n95 and kn95 ones, irrefutably reduce the spread of covid. Washing your hands well and often irrefutably reduces the spread of covid. The science here is icing on the cake and it just tells us, sometimes roughly, how much these things help. Beyond those two things the vaccines are an entirely different story (because thanks Trump), but the science is more clear there, they also help.