I’m interested in creating probabilistic models. Here’s context to frame my question:
I’ve got enough adjacent experience to be able to understand tutorials/examples for simple Turing and Gen models for things like coin flips, linear regressions, and multinomial logistic regressions.
I can write syntactically correct Turing and Gen models, and can to fit, sample and predict.
I can write simple models (that don’t use/require PPLs) using Distributions.jl that sample from a bunch of independent distributions to approximate joint distributions - e.g., a bunch of inputs to a product whose prices fit truncated normal distributions to provide a distribution of the range and likely cost of all the inputs.
With work and patience, I can parse enough of the equations and follow the linear algebra to understand the operations they’re describing. I don’t always have an intuition for what they’re doing.
In short, I am neither a total babe in the woods nor am I coming to this from a discipline where I would have learned the conceptual basics.
With the above as context, this is my question:
How can I learn about creating probabilistic models?
The resources I’ve found tend to be intended for either “hello, world” audiences or for researchers who want to translate their domain to PPL. The resources need not be Julia specific - I can read most syntaxes, so R, Python, Stan, whatever is fine.
This need not be a short/quick path. I’m willing to put in the work.
I would highly recommend reading up measure theory, here’s a basic intro by terry tao himself. As a math major, for me, measure theory founds the basis of probability for me.
Hope this helps
Statistical Rethinking looks great - rave reviews that emphasize the readability and “just right” level of the explanations. I’ve ordered the book.
If this thread ends here then great, I’m already pleased and excited to have this outcome. That said, I’d be very eager to hear other thoughts if others want to make suggestions.
Statistical rethinking is the most prominent resource and likely the best introductory material for non stats & non math people. Reading about measure theory & probability theory is a good strategy if you want to understand the mathematical foundations using a rigorous framework, but will likely not help you in understanding how to define a probabilistic model for your task at hand. You can find Turing models for the book here: https://github.com/StatisticalRethinkingJulia/TuringModels.jl and there is also a zulip group on julialang.zulipchat.com to discuss such matters.
Other books that are often mentioned are: Bayesian Methods for Hackers: Probabilistic Programming and Bayesian Inference and Machine Learning: A Probabilistic Perspective. Apart from those, I recommend reading papers related to the topic you are working on.
Thanks, @trappmartin. As I’ve read this forum, I’ve seen your contributions and wanted to use this opportunity to thank you. Statistical Rethinking looking like a good next book for me after Bayesian Methods for Hackers. Thanks again!
A post-script for people discovering this thread later:
I started re-reading my copy of Bayesian Methods for Hackers while I wait for Statistical Rethinking to arrive. Re-reading it, there is more in the modeling approach sections (Ch1 and Ch2 so far) than I recalled or understood from first reading. So, if you’re like me, and now have the benefit of a little bit of experience, it’s worth re-reading these sections.
Thanks for the update!
Another resource that I can recommend when you feel confident with the level of material covered in statistical rethinking are the case studies from Michael Betancourt:
