The Gibbs sampler is right at the top of my personal list of the greatest technical achievements in statistics. Not only is the asymptotic guarantee of producing a sample from the target distribution a neat and elegant result, but it also opened up a really rich world of probabilistic modeling. From a practical / ML perspective, variational methods are more efficient for getting decent estimates of complex models. And there are a lot of devilish details to MCMC sampling, and the real world _never_ goes to infinity. Still, the sheer theoretical fact that an imperfect, simulation-based method can produce valid statistical estimates is just awesome.