Hacker News Reader

> The paper says Bayesianism is 'aggressive' and 'subjective' while Frequentism is 'cautious' and aims for 'universally acceptable' conclusions. I wouldn't characterize them that way.

Worst-case vs average-case seems like the relevant distinction here. ( http://lesswrong.com/lw/f7t/beyond_bayesians_and_frequentist... http://lesswrong.com/lw/jne/a_fervent_defense_of_frequentist... )

> Frequentist methods try to answer the question of "What is the probability of the observed data" given an assumed model.

Isn't that more null-hypothesis testing...? Frequentism is broader.

I'm not an expert, but my opinion mostly comes from this seminar by Michael Jordan: Are you a Bayesian or a Frequentist? http://videolectures.net/mlss09uk_jordan_bfway/

Around the 10-15 minute mark, he talks about how the two schools of thought naturally arise from conditioning over the two arguments of the loss function, either conditioning over the models or over the data.

(Incidentally, he also characterizes Bayesianism as optimistic and Frequentism as pessimistic. So maybe I'm wrong in that regard above.)

It's a bit of a tangent from your main point, but:

Bayesian methods try to answer the question of "What is the probability of the model" given some observed data

There are people arguing that P(M|D) isn't really a meaningful quantity, because the space of models is infinite, and we can't reasonably put a prior on it. I prefer talking about P(M1|D)/P(M2|D): "The data favors model 1 three times more than it favors model 2"

The philosophical problem with P(M|D) is the same as distinguishing randomness from non-randomness (cf. Popperian falsifiability): there's no test to prove that something is truly random, all we can say is "We've looked for patterns over and over again, but we haven't found one yet".

Also, as a physicist I'm disappointed that the paper asserts that physicists disdained statistics in 1903. For the most part, statistical mechanics was developed in the late 1800s!

Sure, but Boltzmann's ideas were famously controversial at the time. I don't think they represent a good measure of mainstream practice at the time.