PPX: Probabilistic Programming EXecution Protocol and API Based on Flatbuffers
iris-hep.org
iris-hep.org
I had the same feeling with artificial neural networks. I have read a lot about perceptrons, and how they are in a sense universal approximators, but it didn't really click until I have seen a tutorial where they used a convolutional network to classify handwritten digits. I never needed to write a program to recognise handwritten numbers, but could imagine how that can be useful. Also without the tool in question (neural networks) it would have been very hard to achieve the same result.
In short: what is the mnist digits equivalent toy problem of probabilistic programming?
(Disclosure: I use probabilistic programming at Triplebyte, and have previously written a bit about it here https://triplebyte.com/blog/bayesian-inference-for-hiring-en... , though this blog post is really just at the level of Naive Bayes to be easier to understand.)
For your bread and butter PPL starter, I'd go for Variational Autoencoders (VAE). Fun to visualize and tweak: http://pyro.ai/examples/vae.html
If that's too much going on, try this basic tutorial on inference: http://pyro.ai/examples/intro_part_ii.html
The full set of examples is here: http://pyro.ai/examples/
I wish it were that simple! Unfortunately, the only way to really get an intuitive feel for computational probability is going through the exercises.
Before even starting PyMC3, I'd systematically go through some classic problems in probability theory such as the birthday paradox, sleeping beauty problem, 3 prisoner's, etc. And implement them directly in code, without using any dependencies.
Here, for example, is Monte Hall in Golang. You can quickly see how trivial it is to extend the game from 3 doors to 99+ doors. And how a derivation might follow:
https://play.golang.org/p/RGkJ9dToda9
Only after you get a feel for going from probability spaces to events to random variables can you begin to understand the power behind taking human error out of inference that probabilistic environments provide.
Best of luck ;)
Dr. McElreath also posts his lectures on youtube. The R code in the book and his lectures use a library/package he wrote which provides a wrapper to simplify building Stan models. The code has also been translated to PyMC3 on Python.
Code, slides, lecture videos are all referenced here: https://github.com/rmcelreath/statrethinking_winter2019
It might look like a huge amount of content but this course leads you very gently through key concepts, keeping the mathematics to a minimum. Don't be put off if you don't know the R language. The concepts are more important than the programming language and the code examples are kept simple.
If you make it through Statistical Rethinking then you might consider picking up Doing Bayesian Data Analysis by John Kruschke (a.k.a. the puppies book). I've found DBDA to be heavier going than SR but Kruschke takes a different approach to McElreath which can be useful if you get stuck on a concept, need more detail or just want a different angle on the subject.