Comment by cletus | Hacker News Reader

Comment by cletus | Hacker News Reader

His brother is correct: statistics are a group property, or rather, they are they description of what happened with a particular group.

When when you start making predictions you're talking probability not statistics. Statistics may inform probability but there can be issues with this (eg bad sample, insufficiently sized sample, failure to identify the underlying characteristic(s) such that samples are actually flawed and so on).

The problem we have with a species is that we conflate the two. This comes up with highly political issues such as racial profiling. Imagine a situation in a southern border state where statistically more illegal aliens are Hispanic in both absolute and relative terms (meaning absolute number and relative percentage of their given populations). I say this not as a fact but as a non-judgemental scenario to make a point: law enforcement will likely, consciously or subsconsciously, make different suspicions based on such external characteristics.

Is this fair to the individual? Of course not. It borders on an assumption of guilt. People of Middle Eastern descent who regularly fly I'm sure are cursed with this kind of suspicion when almost all of them have done absolutely nothing wrong.

Anyway, as to special training, a relative of mine was married to a guy who went through SAS training (and passed). To those who think this is simply an issue of being sufficiently physically fit, you couldn't be more wrong. Now you have to be sufficiently fit for it to be possible to pass but after that it's a mental issue.

They put these guys through mentally strenuous situations: sleep deprivation followed by interrogation, psychological profiling and the like.

His brother has another point though: on an individual level you should never assume you're going to fail (or succeed for that matter) based on group statistics. Engineering is currently male-dominated but no woman should ever take that as evidence that they can't do it as an individual. Nor should any man assume a specific woman is less likely to be capable, etc.

If you don't think statistics apply to you as an individual, you are essentially claiming to be different from the statistical average in some meaningful way. It may help to articulate what that is.

So maybe most people fail this class, but I think I'm smarter or more determined than the average person, so I shouldn't be discouraged from trying.

I say this because if you don't actually believe you're above average in some meaningful way, you very much should be discouraged by statistics. I don't think myself stronger, more determined, or more desperate than the average navy seal candidate, so even though I'm an individual I shouldn't actually expect to pass the special training that most fail. Similarly, I should expect to pass basic training because most do and I don't think myself any worse.

Whenever you use statistics you are generalizing and likewise the statistics typically don't apply exactly to most people. Everybody is above or below the mean in some way there are almost always data points two standard deviations from the mean... They help us simplify how we think about large populations.

So many little differences can have huge effects on us as individuals that its really only necessary you are different. Failure can only truly be found in seeking to be the same.

I don't think your two racial profiling examples are so much a problem of conflating statistics and probability, as they are a problem of confusion of the inverse (http://en.wikipedia.org/wiki/Confusion_of_the_inverse). It is the case that if you randomly select an illegal immigrant in the US, then the expectation (p ≈ 75%) is that they will be Hispanic. That's not a misapplication of statistics; it's just what the numbers say.

However, that does not mean that if you randomly select a Hispanic person, the expectation is that they are an illegal immigrant. The probability is in fact closer to 15%.

The imbalance of those numbers for people who "look like terrorists" at airports is even more preposterous, of course.

But statistical invalidity is not the only reason that racial profiling is bad. Even if 99% of Hispanic people really were illegal immigrants, it would be unacceptable for police to target them based on race. Not because it would be an invalid conflation of statistics and probability - it wouldn't. The problem is that in order to protect individual rights, the police and the courts must not act strictly according to expectation. We shouldn't imprison someone, for example, just because we reckon there's a 75% chance they were the killer.

And if 99% of Hispanic people were illegal immigrants, and so police hassled every Hispanic person, then we would know just as sure as we'd know that they'd have a 99% success rate, that they'd be violating the civil rights of the other 1%.

You don't even need racial profiling to show that statistics harm individuals. Look at car insurance - in most countries you will have a massive premium to pay,just for committing the crime of being under 25. Even if you are 24, had your licence for 6 years, drove hundreds of thousands of miles without an accident, your insurance will still be higher than that of a 26 year old person who has just passed their drivers test and doesn't know anything about driving. Why? Only because "statistically" you are more likely to be in an accident. Well, so are men, yet it is illegal to make their car insurance higher because of their gender in almost all EU countries. You may not have done anything wrong, but are still "punished" for being part of the statistic. It makes me really upset sometimes.

Well, they also help individuals. Think of the person who is 26 years old, has just passed their drivers' test, and then totals their car and kills the other driver. Insurance shells out a few million in liability claims. Their insurance will go up, obviously, but they're not going to be financially wrecked, and it's the rest of society that bears that burden.

The way to think of it is to think of your whole life statistically, as a series of events where you win some and you lose some. And then don't get upset over the ones you lost, because there are many others where you ended up with the winning end of the deal and you just don't remember it because humans are wired to dwell on losses more than gains.

Sorry but I just cannot agree with your first statement - even if an 18 year old driver causes a crash and kills the other driver, the insurance company is still going to pay 100% of the claim, unless the claim goes over the insurance limit on your specific policy(which in EU is unlimited for death and bodily injuries). So I do not see how paying more when you are under 25 is a benefit.

I thought that, too, with car insurance for an under 25 year old... The price difference actually was about 25 euros a year (Germany)

I am from Poland and it's similar there - the difference is negligible. But I am studying in the UK at the moment and the difference is exorbitant - I would pay almost 3000 pounds per year for insuring my 2.0L Peugeot Partner, and my friend who only passed his test few weeks after turning 26 is paying 400 pounds per year for insuring his 1.6L Golf. Just because he is older, and not because he is a safer driver.

Nothing wrong with stats. They are what they are. Problem is when we then apply them to individuals. Then we are unfairly stereotyping the individual.

Car insurance is the prime example of unfairness and being punished for the actions of others, purely because of statistics.

Lots of care insurances provide benefits to people that deviate from the statistic minimum in a meaningful way - few miles per year, ... In all other ways, yes - you get to pay an price that's averaged over a reasonably large peer group that you just happen to fall into. That's how insurances work: They redistribute risk. They protect you from extreme outcomes by charging a low, steady price. All in all, you usually pay more than you get out and only a few actually benefit financially. Take a look at large car rental companies: They don't actually insure their cars against theft. They're large enough to do the risk arbitration themselves. So I guess you just need to find a large enough pool of people that agree with you and then form your own insurance - with lower fees.

Reply on news.ycombinator.com