Say we have data X, and some non-finite dimensional index into the family of functions that describe the the data, called \theta.
The Bayesian perspective classically holds \theta constant and optimizes the expected loss, conditioned on the data X. The frequentist perspective, on the other hand, classically optimizes \theta, that is, it picks the best \theta over the data X, unconditionally.
This has two impacts. First, all things equal frequentist statistics will tend to be more stable, and more calibrated, but less coherent. It is commonly said that frequentist statistics will "isolate" one from poor decision making, and all things equal, that will be true.
Specific, clear wins for frequentists are bootstrapping procedures (e.g., Efron's bootstrap, the b of n bootstrap, Jordan's own scalable "bag of bootstraps" from NIPS 2011), which are methods for building what are called "quantifiers" for "estimators". In short, this means that if you have some estimator (e.g., a classifier, or a mean, or whatever), you want to be able to quantify the certainty of your estimator -- so if you've only seen 5 examples, you want to express that you're less certain about this. This is clearly a frequentist application, not a Bayesian application, and in general, it points to the fact that pure frequentist tools not only have a place in inference, but they fill a niche that Bayesian tools necessarily will not, and in some cases, cannot, fill.