I wouldn't contrast it with machine learning or deep learning. Probabilistic programming is focused on building languages or libraries that incorporate fundamental probabilistic building blocks, model building statements and inference strategies as first class citizens within the language. Its been remarkably successful, check out Stan, Pyro, PyMC, Tensorflow Probability, JAGS, BUGS and Turing as very successful projects that have been used to tackle challenging and diverse problems with probabilistic modeling . The more modern probabilistic programming languages are actually designed to incorporate the advances of deep learning by making it easy to embed ANNs into models by using them to parameterize random variables, Pyro and Tensorflow Probability are probably the best examples of this.

note that these are not exclusive. you could divide ML into a traditional statistical approach and a probabilistic one that is concerned with deriving the underlying probability distribution. probabilistic programming is kind of like a domain specific language for achieving this. there is also differential programming that works on the same principle. there are certainly industrial usages of this paradigm. look up pyro (http://pyro.ai/examples/intro_part_i.html) for ppl and jax (https://github.com/google/jax) for differential programming

It's the standard tool for building Bayesian statistical models, which aren't glamorously crushing SotA records on AI-like prediction tasks, but are valuable in both academia and industry.

Not an either-or situation. Many ML algorithms can be formulated in a probabilistic way, and PPLs can represent them. But the choice of the underlying inferential algorithm (MCMC, variational, etc) greatly affects the scale and type of solvable problems, so it's not just the language itself.

PPLs using MCMC (e.g., Stan and pymc) are first-choice tools for Bayesian inference on small to medium data analysis, which is lots more common than Google-scale data analysis.

My professor in school years ago was working on probabilistic hardware and showing pretty good thermal benefits. It’s a drastically different way to build software so I don’t know how commercially viable it’ll become until someone shows it has a drastically better power per watt ratio (like 1000x or 10000x, maybe higher to account for the cost of transitioning existing software)