louden

Tree based methods, such as Random Forests, provide some nice properties at the expense of interpretability. Trees by themselves tend to be fairly variable.

Not everybody needs to know how to do math beyond arithmetic (though the opportunity should be provided).

The focus should be making sure everybody can understand the math, and especially statistics, they are presented with everyday.

Even with those other methods in place, the leaderboard would still favor methods that overfit. That is why the final score is determined on a dataset (the validation set) that is not used until the model is locked down. The public leaderboard is based on the test set.

A more in depth explanation on the use of three datasets for model building can be found here: http://stats.stackexchange.com/questions/19048/what-is-the-d...

To be fair to Stripe, this will be an issue regardless of the provider you use. They share the same back-end partners. As long as a business appears to be high risk, the risk of shut down by payment provider will exist.

This has been my experience as well. To combat this, I always start with a written analysis plan that forces me to think out all of my analyses.

If you have time, in the long term I would recommend looking at the theory behind the methods. It will give you a lot of insight on why a person is using a particular method and when it is inappropriate to use a certain method.

I second using kaggle. They have a lot of 'competitions' that are really just learning exercises with a lot of information about the techniques used directly linked to the competition.

This article illustrates the problem with over-fitting a model even when some data is withheld for testing. This is a trap that one can fall into when using training and testing sets.

It's a trade off. Clopper-Pearson can be overly conservative in many instances. I tend to use Jeffreys interval personally, which is a Bayesian method.

Brown et al. give a formal treatment of the subject with simulations showing actual coverage probabilities for the various methods. It is worth a read if you want to really dive into the subject.

ref: Brown, Lawrence D.; Cai, T. Tony; DasGupta, Anirban (2001). "Interval Estimation for a Binomial Proportion". Statistical Science 16 (2): 101–133

R does support functional programming. In fact, it was based on Scheme (see discussion for comparison: https://stat.ethz.ch/pipermail/r-help/2008-December/181982.h...).