521 karma · joined May 3, 2019
Sample numbers:
* Population of US: 3e8
* People that have COVID right now in the US: 1e5 (30x the confirmed cases right now)
* Disease rate: 3e-4
* FALSE POSITIVE RATE for COVID test: 1% (likely higher)
* TRUE POSITIVE RATE for COVID: 100% (definitely lower)
If you take the test, don’t have symptoms or direct contact, and then the test comes back positive for the disease, then under these pretty reasonable assumptions, the likelihood you have the disease is
P(actually have COVID| positive test) = 3e-4 /(1e-2(1-3e-4) +3e-4) = 2.9%
* assume false positive rate of 10%, assume true positive rate of 100% (it's not but let's be generous)
* maybe 1/10 of the population (30 million) gets tested
* lets say 100,000 people have COVID right now (50x the official number)
number of positive tests = 30 million * 0.1 + 100k * 1 = 3,100,000
fraction of postive tests that actually have the disease = 1 / 31 = 3.2%
problem is FPR (1) depends on the population tested (i.e. p(covid | positive test) != p(covid|positive test, some symptoms) != ...), (2) we need very accurate measurements of FPR because:
lets say we constrain FPR to 10% +/- 1% ==> 10% uncertainty in FPR -- that means our inference of the number of infected people is:
n_infected = (n_positive_tests - FPR * n_tested)
which is: -200,000 to 400,000
so...not very useful.
> suspending evictions, delaying mortgage payments, nationalizing healthcare
I mean sure, I don't think I would disagree with any of that at all (good luck with the last part though, we've been trying that for 40 years), but to say that this wouldn't have a detrimental impact on the economy is a bit confusing to me...it will and that will hurt, but the idea is that it will hurt less than something like COVID running rampant or without measures to protect workers who will be hurt most by the economic fallout.
If FPR is 10% and we test 100,000,000 people in the US but only 100,000 have the disease right now, then assuming a TPR of 100% (and a FNR of 0%) we will have 10,100,000 positive results = 101 x the number of people with COVID? How is that useful?
Disclaimer: I am maybe an idiot.
The grad students have a tradition (or used to) called “Thunch” (Thursday Lunch) where they invite someone to come have lunch and hang out every week. It was by far one of the highlights of my life — maybe 10 people in the room max, eating pizza, shooting shit with some of the most impressive and accomplished people you could hope to meet — Nobel prize laureates, Neil deGrasse Tyson, famous authors, whoever would respond to our invitation (people tended to say yes). Dyson was one of them. At the time he was already like 92 years old, dressed in a suit, occasionally using a handkerchief, and this guy had just published a result in Nature. The guy was SHARP at 92. Nicest, most interesting person I have ever met. Knew Einstein from his early days at IAS. Worked on an early version of the space program that looked into launching people into space with large spring loaded contraptions that were to be propelled with nuclear weapons. Those were the days I guess...
So sorry to see he’s gone, the guy was a living legend.
Now that I've taken some time to actually READ the article referenced in this post, I have a couple more things to say; basically: I agree with virtually everything that they're saying in the paper (except that I still don't understand why Bayesian stats failing in QM says absolutely anything about the validity of Bayesian stats), and I think I set up my own straw man here a bit. I thought this was another "Bayesian stats is bad because priors" post, which in a way it SORT of is, but their arguments are a bit more on the side of "you should take 'Bayesian' analyses with a large grain of salt sometimes because...well...priors" <- and THAT I completely agree with.
The only beef I have with these types of articles are when they seem to imply that because priors sometimes have an outsized effect on the results, we should not use Bayesian statistics/analyses. But really, there is no other option. If you have a statistical question, you're stuck with having to answer for your priors. That being said, what IS good advice is to not simply believe every 'Bayesian' conclusion for exactly that reason -- that result is only as good as the (1) prior, (2) model assumptions, and (3) the data. (2) and (3) are easy to fight about, but (1) can be hard to evaluate, and easily can lead all of us astray even when the prior sounds reasonable (flat priors).
I absolutely agree with you. Priors are the (very difficult) job of the practitioner.
> choosing priors is a part of statistics
Ah so I think I agree, maybe would nitpick and say priors are part of statistics and CHOOSING priors is part of modeling maybe? Not totally comfortable with that characterization either it’s just that “priors” and “choosing priors” are two different things.
> therefore Bayesian statistics has a “hole” in it.
Completely disagree; this is my main understanding of the argument against Bayesian statistics, I just don’t understand how this follows from the last statement. Priors are hard. But they are explicit! And a necessary ingredient of the posterior. It’s not like we can say “oh priors are hard, so let’s not do Bayesian stats but instead let’s do X” what is X?
Bayesian stats is nothing more than a rigorous way to transform beliefs + data into a posterior. Yes, flat priors are not always the best choice. That’s not a criticism of Bayesian stats. It’s a statement about how actually formulating a prior is often times the hardest part of a problem. Is Bayesian stats useful for describing or understanding QM? Idk, again, not really it’s job...
Use Bayesian stats, not with an air of suspicion, but a respect for the fact that it will give you the results implied by your data and prior, under your model assumptions. Nothing more, nothing less. By the way, what is the alternative if you find yourself in a situation where your result depends strongly on your prior and you aren’t really sure how to choose your prior? Wave your hands and find an ad hoc frequentist approach? How about just admitting to yourself that your data isn’t enough to make up for the fact that you can’t really quantify your true prior belief?
If you read this and disagree, I sincerely implore you to comment — I just don’t understand the “debate” aspect of Bayesian stats. Some people misunderstand what Bayesian stats is sometimes (very understandable) but I have yet to see a legitimate philosophical or mathematical critique of the Bayesian approach that really made any sense to me. I would like to know if I am wrong though...
If you can generate a single static figure, you can manually generate animations by generating each frame separately using your favorite plotting software. You can then stitch them together using many different tools (I use matplotlib) to generate an animation (video, gif, etc.).
What language do you like to use? What's your background -- are you a Matlab person?
Non-trader here, what do you mean by a particular trading strategy "harm[ing] a market," how would you actually measure harm to a market by a given trading strategy, and would whatever metric you decide on for that not just be a dressed-up subjective argument?
That being said, Bayesian statistics is not SUFFICIENT for doing data science well. You need to know pragmatic solutions to intractable problems, what heuristics might be ok to try, how to EXPLAIN things to people, how to code well enough, etc. But boy does knowing Bayesian statistics help.