What they found was a correlation between false positive rates and true positive rates, where IF you could accept a false positive rate of 0.1%, THEN you could get a little over 50% true positive rate with the longest seam length under "ideal imaging conditions".
They do not attempt to estimate the false positive rate of whatever methodology FBI agents are using in practice. I doubt the FBI has a rigorous well-defined test that could be analyzed; I suspect this is just a general technique that is applied by different agents in different ways.
Even if the FBI's methods approach this ideal false positive rate (which they can't), they are not describing it honestly in court.
> When, for example, an FBI analyst identified the plaid shirt in a surveillance video as that belonging to Wilbert McKreith, the analyst stated in court that the odds that two different shirts would match were a staggering 1-in-650 billion. The analyst arrived at this number by making eight measurements along two seams, estimating the probability that the plaid stripes at these eight locations were misaligned by a certain distance, and then multiplying all eight of these probabilities to reach the astronom-ically low odds of 1-in-650 billion.
>The analyst excluded any seams that did not satisfy each of the following properties: 1) the length of the seam was at least 30 cm, 2) the resolution of this 30-cm seam was at least 1,500 pixels, 3) the width of the seam was at least 30 pixels, and 4) the final extracted 1D signal clearly showed a distinct pattern of ridges and valleys.
Even with photos taken specifically to see these patterns, "[e]nforcement of these criteria reduced the original 211 left/right and inner/outer seams to 164 left/inner, 162 right/inner, 81 left/outer, and 90 right/outer seams."
Unfortunately they aren't very clear on what they mean by different orientations/lighting conditions/drapes.
1 person committed a crime. So the true positive rate is 1 out of 7 billion or so. So how many false positives would 0.1% lead to? About 7 million pants wearers?
And for false negatives, we expect half the time the true criminal's pants would prove them innocent?
I don't think you can really trust evidence if the true criminal has a coin-flip chance to show innocence with it.
https://www.ncbi.nlm.nih.gov/m/pubmed/12423006/
Without confirmation, false positive rates varied from 2 to 4 percent. That means up to 4 percent of test subjects will end up arrested because of a false positive.
If a piece of evidence is used to identify, you select for the population that matches that property.