1. Typically very accurate. 2. Sound theory and good uncertainty estimates. 3. Being a Bayesian model, tuning is very easy.
The main competing models for some of the same tasks are gradient boosted decision trees and sometimes neural networks. GBTs win over NN for most tasks in practice, although they don't get much hype. GPs do well with smooth data in my experience, with GBTs winning over any data where a limited number of bespoke decision tree splitting rules can represent the data well.
Interestingly, damn near anything, including neural networks, linear regressionm and GBTs can be interpreted as gaussian processes (or an approximation of GPs) by certain choice of covariance function. GPs are just functions in a reproducing kernel hilbert space defined by the covariance function. That can include most anything.
GPs with full covariance matrices don't scale to more than a few thousand examples (n^3), but approximations can be made to scale to large datasets.