Hacker News Reader

Comment by eis | Hacker News Reader

> Millimeter-wave machines at airports today are totally and completely useless. Germany's transportation administration reported a 54% false positive rate for these pieces of rubbish. You are better off randomly searching people by flipping a coin.

Edit: note that it seems I had a wrong understanding of the term false positive rate, ignore this comment.

Statistics doesn't work like that. The false positive rate is pretty meaningless in this context. It only determines how much useless manual work is performed after the machine flags you incorrectly.

You should look at the false negative rate in order to determine how much of a security plus they bring.

That being said, I think the whole airport theater is ridicolous. If terrorists wanted to kill people, they literally have thousands of alternative ways to achieve that.

View on HN

Not quite true. A high false positive rate leads the people running the machines to stop reacting seriously to alerts, which means that they don’t do the manual checks with as much care. This is even more of an issue when everyone in line knows they are bullshit and has no respect for the people running them.

A false positive rate of close to 50% means that every second alert indeed pointed at something real. That should not cause the staff to get unmotivated. Again, without the false negative rate we can't judge if they are complete bullshit or not. If it were a 90% false positive rate then sure, that would clearly have the effects you outlined. I don't think at around 50% that's the case yet.

> A false positive rate of close to 50% means that every second alert indeed pointed at something real.

No it doesn't.

Please see my other comment which references the definition of false positive rate because I think your understanding of the term is wrong. You have a 50% alarm/trigger rate in mind. And please try to reply with a bit more substance than "No".

To quote an article on the subject: "meaning that every other person who went through the scanner had to undergo at least a limited pat-down that found nothing"

> And please try to reply with a bit more substance than "No".

Eh. I was hoping you'd just delete the comment rather than have two parallel arguments.

> The false positive rate is calculated as the ratio between the number of negative events wrongly categorized as positive (false positives) and the total number of actual negative events (regardless of classification).

https://en.m.wikipedia.org/wiki/False_positive_rate

54% FPR means half of all people who aren't trying to bring anything illegal onto the plane gets stopped.

Since basically nobody are actually trying to do this, that also means that more than half of all people going through gets stopped.

The ratio of true positives to positives must thus also be vanishingly small.

No, it means that about 50% of people scanned would trigger a "positive". The false positive rate is the number of false positives divided by the number of actual negatives.

100 people, 50 false alarms, 1 actual problem alerted and 49 marked as safe correctly = 50/(50+49) = ~50%.

That's just wrong. This is ML 101

False positive = (Machine says positive but grand truth is negative) / (Machine says is positive)

In English, a false positive is when the machine declares something as positive but it is actually negative.

This fits a sanity test because ~50% of people that pass through a detector do not get patted down. The truth is closer to 10%. Even then, a true positive probably includes someone with two pennies in their pocket.

https://en.m.wikipedia.org/wiki/False_positive_rate

In statistics and ML it’s (FP) / (FP + TN) aka (false positives) / (actual negatives samples). This is by far the most common definition.

Here ML using the common definition see fall-out: https://en.m.wikipedia.org/wiki/Precision_and_recall

You are right this is ML 101, but you are wrong. Someone else has linked the wiki page, and I've posted it elsewhere and I would strongly recommend you read it as your interpretation is very incorrect.

> In English, a false positive is when the machine declares something as positive but it is actually negative.

Yes, and the rate is based on the frequency this happens in your negative set.

>This fits a sanity test because ~50% of people that pass through a detector do not get patted down.

The 50% figure may be wrong, but my definition of what a false positive rate is not.

You're thinking of FDR: False discovery rate. It equals Σ False positive/Σ Predicted condition positive.

See https://en.m.wikipedia.org/wiki/Precision_and_recall for a table of different terms.

Wow, that definition is counter-intuitive. Ive expected the false-positive-rate to be the number of false positives divided by the number of all positives. Isn't it how "rates" are usually defined? No wonder everybody is confused.

I think it's a perspective thing. You are looking at the rate of false results, so given a particular type of input what rate do you get it right? For negative ground truth your possible outputs are "false (positive)" and "true (negative)".

I agree it can be confusing, but I think people would also make the opposite mistake if it were defined differently.

You're thinking of FDR: False discovery rate. It equals Σ False positive/Σ Predicted condition positive.

See https://en.m.wikipedia.org/wiki/Precision_and_recall for a table of different terms.

In this context of using them in public spaces it’s the false positive that matters most. The complaint is that the machines basically always alarm, which means the only thing they can be used for is making up a bogus excuse to manually search / harass someone based on no real suspicion.

But they don't always alarm according to the number that was presented. It alarms twice as many times as it should. That's not good and I certainly wouldn't want these in public spaces.

But you can't just look at the false positive rate and how much more work and unpleasant/unnecessary searches they cause. You have to take into account the false negative rate to get a complete picture. Manufacturers and the people who want these machines will always err on the side of too many false positives than too many false negatives.

Again, I personally think that these machines have no place in public spaces, no matter their hit/miss-rate.

> It alarms twice as many times as it should.

That's not what a false positive rate is. It alarms on half of everyone that goes through. It's utter garbage as a first-line test.

I think this is a misunderstanding. From Wikipedia [1]:

> A false positive error, or in short a false positive, commonly called a "false alarm", is a result that indicates a given condition exists, when it does not.

So a 50% false positive means also that in 50% of alarms the condition was actually true.

What you mean is a 50% alarm/trigger rate. Which, depending on how many people do have something on them that should trigger the alarm, would result in a far far higher false positive rate.

1: https://en.wikipedia.org/wiki/False_positives_and_false_nega...

> What you mean is a 50% alarm/trigger rate.

No I don't. It's just that the true positive rate is close enough to zero that both false positive and alarm rate are roughly the same number.

Nevertheless you're still not understanding "false positive" correctly. If you correctly pick 5 out of 100 true positives without any flaws, and also have a 54% false positive rate, you're grabbing 51 false positives, and >90% of your picks were incorrect.

And I'm pretty sure the true positive rate is well under 5% to start with, which just makes it worse.

"In Germany, the false positive rate was 54 percent, meaning that every other person who went through the scanner had to undergo at least a limited pat-down that found nothing."

This is wrong. Even your quote shows this. A false positive rate is how often it triggers when the true result is false.

> A false positive error, or in short a false positive, commonly called a "false alarm", is a result that indicates a given condition exists, when it does not.

So 50% false positive rate means 50% of people who should not be flagged are flagged.

https://en.wikipedia.org/wiki/False_positive_rate

It is quite definitly true that every second person needs to be patted down without a reason, purely on my personal experience as a frequent flyer

This is what they do in Mexico when entering the country from the perspective of flipping a coin. I took my vacation in Maya Riveria and upon entering the country they have a box with a button. You push the button and it lights up green or red. Green means you go, red means you have some additional security checks.

Reply on news.ycombinator.com