Sample size and power for rare events
win-vector.com
win-vector.com
I use Bayesian methods simply to restrict the test space. It's also relatively simple (computer is doing the math, essentially).
Edit: I've added some discussion on what you would expect in the Bayesian case as an comment on the blog.
It it just a matter of taste with differing opinions on the correctness (or morality?) of the guessing that's necessary for Bayesian analysis? Can we not just figure out the error as a function of the wrongness of the guess and be done with it? Are there guesses so wrong that no amount of data is sufficient for Bayesian and Frequentist analyses to converge?
Efron says the difference is Frequentists assume the quantity to be modeled is unknown but has a single fixed value (so you can't make probability statements about it) and the measurements have some possible variability. Bayesians model the unknown with an explicit prior distribution for the unknown quantity to be inferred and assume the measurements are fixed. So each school means something different when they state a probability (Frequentist variation over data draws, Bayesian variation over the unknown to be measured).
Gelman defines Bayesian methods as the proper setting up of a generative model and computing joint probabilities of parameters and data. In his view Frequentist don't compute joint probabilities but at best conditional probabilities (which assume out modeling likely values for the unknown quantity to be inferred).
The joke of the comment is: both sides tend to snipe on the other (as neither framework looks good if you assume the other).
My partner has a nice article on the topic: http://www.win-vector.com/blog/2013/05/bayesian-and-frequent...
Your comment and that article really help convince me that there exist people who actually understand what's going on.
A rough summary:
Frequentist methods tend to try to characterize the question of interest in terms of a single measurable statistic. This is particularly useful in circumstances where memory/cpu is tight (i.e. every single circumstance before the last few decades.) This necessarily means though that these techniques lose much of the information in the data. They also try to make very few a priori assumptions about the underlying data. The bayesian critique of this is that the assumptions are still there, just hidden, and as a result are much more likely to be wrong.
Bayesian methods tend to try to characterize the question of interest in terms of the entire distribution of outcomes. This causes there to be little information lost between the data and the conclusions, but is often computationally intractable in common circumstances. Bayesians tend to make strong a priori assumptions about the form the data will take, and to ameliorate this they make their assumptions very explicit in the model. The Frequentist critique of this is that every time you read a bayesian piece of analysis, you have to be very careful to determine whether the outcome described is merely the assumed prior + noise.
Both cultures provide essential mathematical tools for any statistician's toolbox, and it's often just a matter of taste which get used. Hence the bike-shedding.
I get how bayesian probability works mathematically, but doesn't it rely on having accurate prior measurements for the likelihood of an outcome? That is, working from prior probabilities only becomes useful and accurate if prior probabilities are accurate to begin with.
"That is the likelihood generally dominates"?
"My results are generally very robust to choice of prior parameters"?
Neither of those sentences are comprehensible.
http://www.chrisstucchio.com/blog/2013/bayesian_analysis_con...
I'm talking more to the folks over on LW who say things such as, "It's 17% likely that my actions will cause harm, so I'll act./won't act."
I believe that's what RA_Fisher was describing. That exact situation happened to me and my cofounder a few days ago. I'll be blogging it soon.