I've used Gaussian Processes to great effect in the field of energy regression/forecasting for commercial buildings, and have found them to be generally superior to other approaches due to the richness of prior information you can encode into the GP kernel. I'll be attending
http://gpss.cc/ this year as well, so you could consider me something of an evangelist!
For example, you might approach a regression problem with the information that you expect the outcome to vary very smoothly w.r.t covariate 1, periodically (cyclically) over a long timespan w.r.t covariate 2, and periodically and non-smoothly (but with a known period) w.r.t covariate 3 and 4.
You don't know the exact form of these relationships (ie: what kind of periodicity exactly, or the nature of the 'smooth' relationship), but you're pretty sure they are related in that way.
GP regression allows you to draw from a posterior distribution over a function space of functions with those properties, conditioned on the observed data. The resulting credible intervals (HPD) are directly interpretable and meaningful without hand-wringing. And in a much more practical sense, the results you get tend to be extremely good compared to other techniques (trees, neural nets, GAM's, etc).
Other very useful applications of GP's include hyperparameter optimization, classificaton, and GP-latent variable models (GPLVMs) for dimensionality reduction and manifold embeddings. The only downside, which other users have mentioned, is the time-space complexity of inverting the covariance matrix, making the computation O(n^3). Various approaches to making GP fitting more performant, including sparse GP models with inducing points, variational approximations to the posterior, help a bit.
My personal feeling is that these models fall squarely into the category of 'clearly the best option if it weren't for the computational complexity'. If that bridge is crossed, we're going to see their popularity grow dramatically.