Hacker News Reader

Comment by spywaregorilla | Hacker News Reader

I find it funny that this joke seems to imply getting 5/12 false positives means that it's an obviously bad model.

I'm not a statistician, but if something is ~50% accurate in predicting a recession does that not make it roughly equivalent to a coin flip?

It's a big topic. But 5/12 false positives tells you nothing about accuracy. You also need to measure the number of true negatives and false negatives and what not. Accuracy tends to be worthless for most use cases when the event being predicted is rare. If you're predicting a 1 in a million event always predicting NO gives you a 99.9999% accurate model.

Many medical tests are 99%+ accurate, but the false positive rate, meaning the probability that you don't have the disease given a false positive test is still extremely high. Like 90%+. Bayes law fucks us all.

For those interested, a bit more background on false positives of rare events

https://math.hmc.edu/funfacts/medical-tests-and-bayes-theore...

Medical tests are often designed that way. For example cheap/quick hiv tests have a low false negative and a high false positive. They are still useful, you filter out most people safely and the ones with a positive test can take a more expensive test to check.

Yes, but also no. If your test only flags a false positive 1 in a million times, searching for a disease that affects 1 in 100 million people, you'll get a ratio of 100:1 false positives. Still very useful as you explain, but I think it's misleading to imply that's what they're aspiring to. It's hard to make a test that never returns false positives.

I don’t disagree with you, merely said that for some reason it was possible to make a cheaper and quicker HIV test that has a high false positive and a low false negative.

I have myself tested positive twice without actually being positive when being deployed with the army, so agree on all your points that this is not great and not an intended goal,. Just what we get when we try to make the cheapest possible test today.

I can't tell how serious, but just in case.

Probability of recession ('how often do you check?' question aside) is fairly low. If you can guess what specific number comes up on a dice roll 50% of the time, you're good at predicting dice rolls.

> does that not make it roughly equivalent to a coin flip?

No.

Intuition: predicting the outcome of a coin flip correctly 50% of the time is unimpressive, but predicting the outcome of a D20 dice roll correctly 50% of the time is incredibly impressive.

In this case the outcome is binary though, "Recession" or "No recession"

It's not binary, it's "is the recession going to happen in the next few months/years", so the prediction is a number. More like D20 in one of the neighbour threads. But not necessarily a D20, could be a D6 so less impressive.

Rollong a 20 vs not 20 is binary, but that doesn't mean it happens 50% of the time.

Recession or not is binary, but that doesn't mean we have a 50% chance of entering one on a given day.

Over thousands of periods. And each time the prediction was no the outcome was no, and among the few times the predictions was yes it happened 50% of the time.

Recessions are forecast on a quarterly basis, so not thousands, but more than 20 between each recession (typically, historically). That's why I chose a D20 -- out of coin/dice/D6/D20/Roulette, it's the RNG that's most reasonably close to the number of quarters between recessions.

How long have we been forecasting? More than 50 years or more than 200 quarters. 5/12 would be 7 misses out of 200, that means it was right 193 out of 200 times.

That doesn't include false negatives of course. Actuals vs positive predictions tells you nothing interesting.

Yeah there are many additional reasons this is different from a coin clip, but the confusion of predicting correctly 50% of the time in a 50/50 event versus 50% of the time for a 1/20+ event was the most glaring issue so I started with that one...

The sides of the dice are financial quarters. The roll is the financial quarter that a recession starts.

If recessions happened every other financial quarter then the outcome would be binary. But historically recessions not that common; they happen in less than 1 in 20 financial quarters. So the D20 analogy is actually something of an understatement.

It’s binary, but not balanced. Specifically, it’s predicting the beginning of a recession, and very few time periods are the beginning of a recession.

So it’s like predicting that you’ll roll a 1 on a die. Flipping a coin for your prediction would say you’ll roll the 1 50% of the time, which is obviously wrong.

"Precision" is the right word, that's the probability that you identified a recession but there wasn't a recession.

You can get very high accuracy (more like 95%) by predicting that there will not be a recession because recessions are less common than expansions.

An indicator that flashes occasionally but it even if it wrong half of the time (you do the coin flip after the warning) it is giving useful information about an unusual event.

it's a _necessary_ but _not sufficient_ condition for a recession. so it does provide you information. if the yield curve does not invert, it is unlikely there will be a recession.

It really depends. If you count 100 times where the yield was not inverted and no recession was predicted, the odds get a lot better.

What type of bet you make with the market matters a lot. If your play is very convex and it gives you the opportunity to risk $1 to make $10 then that's a worthwhile bet to make provided that losing $1 10 times doesn't wipe you out. Since market crashes tend to be violent and volatile there are instruments out there that benefit from higher volatility and can give you 100x returns.

So when you combine a 5/12 accuracy indicator with a play that is low probability, little fee to entry, but huge reward you have a decent strategy.

Only if the probability of the true positive is 50%. If I had a model that told me with 50% certainty that a black swan event was about to occur, that would be pretty valuable.

If you flipped a coin every day to determine if a recession started, you'd have a way worse record than 50/50.

Well if your coin says "no recession" on one side and "recession" on the other, you'd get 50/50.

Only if you throw it twice as many times as there are recessions.

if you throw a coin every day, and it's 50% accurate in predicting recessions that only happen every few years, that's a hell of a signal.

Reply on news.ycombinator.com