# Introductive material on Bayesian Networks inference / probabilistic programming?

Hello, any (gentle) introductory material to suggest on the topic of Bayesian Networks and probabilistic programming ? Or knowledge I need in advance in order to understand it ?

Context: I have clear the concepts of distribution, random variable, joint/marginal/conditional, Bayesian theorem and also prior/posterior distributions and likelihood, but I am still at odd to understand even the objectives of probabilistic programming or how Bayesian networks can be estimated from data and how predictions can be generated.
The Turing.jl example and documentation is already too specific for me (the prologue â€śIf you are already well-versed in probabilistic programmingâ€ť unfortunately doesnâ€™t apply to my case) and books like â€śAn Introduction to Probabilistic Programmingâ€ť seems both too wide and too theoretical (I canâ€™t even understand the notation on the first few pages)â€¦
On the wikipedia page on Bayesian Networks I can follow until they start dealing with interventional questions, than I am lost (by the way, no reference to Turing.jl is given in that page)

Are probabilistic programming languages â€śtoolsâ€ť to express and fit a Bayesian Network ?

So, you can really see I need a very â€śgentleâ€ť introduction on the topic (conditional to my â€ślevelâ€ť ) !!

2 Likes

One of the nicest introductions in my opinion, with practical examples:

1 Like

Thanks â€¦ nice and really from the basicâ€¦ a pity it has been ported to TensorFlow but not on Turingâ€¦ ok, I guess that after it it will be relatively quick to move knowledge from PyMC to Turing

1 Like

There is a (partial) port of Bayesian methods for Hackers to Julia.
A good introduction to Bayesian networks is the chapter on graphical models in the ML book by Bishop. While some earlier attempts on probabilistic programming had been based on graphical models â€“ exploiting their structure for inference by message-passing â€“ modern sampling methods â€“ such as NUTS implemented in Turing or PyMC â€“ are more general and only require a differentiable function computing the log joint probability \log p(\mathrm{data}, \mathrm{parameters}).

2 Likes

Statistical Rethinking is well-regarded for Bayesian inference.

3 Likes

In addition to seconding Statistical Rethinking, I recently found this great explainer of belief propagation on Bayesian networks.
https://gaussianbp.github.io/

1 Like

Ok, I have now two main options to start with, â€śBayesian methods for Hackersâ€ť and "Statistical Rethinking ".

Who knows them both, how do they compare, including in scope ?

I know both and wouldnâ€™t recommend statistical rethinking as an intro book.

I liked this: Probabilistic Models of Cognition

https://probmods.org/

2 Likes

I know them both and my recommendation is always Statistical Rethinking. Itâ€™s really easy to read and gives you a good overview of Bayesian modelling and how to express them in a probabilistic programming language. I also enjoy the humorous tone in the book.

1 Like

There are lots of source about Bayesian Stat
here is a brief list, not all

## Book

How to become a Bayesian in eight easy steps: An annotated reading list

the author has a good excellent blog about basic concept of bayes stat ,here it is : https://alexanderetz.com

Bayesian Reasoning and Machine Learning | Higher Education from Cambridge

Bayesian Cognitive Modeling

## Turing.jl

If you want to learn about Turing.jl package, you can reference

Another partial list

Iâ€™d start with Frank Woodâ€™s presentation https://www.cs.ubc.ca/~schmidtm/Courses/540-W18/wood.pdf , which gives a great overview of Probabilistic Programming (no need to go over all the 82 pages, and note that the language for the examples is Anglican). Frank Wood is one of the authors of the book â€śAn Introduction to Probabilistic Programmingâ€ť you mentioned. And Iâ€™d then read the first chapter of that book, skipping whatever seems over your head on a first reading.

Much easier than â€śAn introduction to probabilistic programmingâ€ť, and with examples you get to play with right away, is the fantastic â€śProbabilistic Models of Cognitionâ€ť, https://probmods.org/, mentioned by @hackandthink. HĂĄkan Kjellerstrand has written many of those models in Turing in My Julia Turing.jl (probabilistic programming) page, another great resource already mentioned by @math4mad. (Note, though, that at the time HĂĄkan wrote his code, I think some facilities now available in Turing did not yet exist, or were as well documented, so a few things might be doable in a slightly more idiomatic â€śturingesqueâ€ť way as of October 2023).

A comparison of some Bayesian statistics books, including â€śStatistical Rethinkingâ€ť (as well as â€śA Studentâ€™s Guide to Bayesian Statisticsâ€ť, â€śBayesian Data Analysisâ€ť, and â€śRegression and Other Storiesâ€ť) is available here

(not everybody agrees with everything said there, but I think it is a balanced overview. Oh, and if you decide to give BDA a go, Aki Vehtari has a course based on it: Bayesian Data Analysis course )

Anyway, Iâ€™d definitely start with Frank Woodâ€™s presentation and the first chapter of their book: it will answer some of your questions (â€śthe objectives of probabilistic programmingâ€ť) and help you anticipate and understand the differences of focus and emphasis between, say, â€śStatistical Rethinkingâ€ť and â€śRegression and other storiesâ€ť on the one hand and â€śProbabilistic models of cognitionâ€ť (or â€śAn Introduction to Probabilistic Programmingâ€ť) on the other.

1 Like

Tutorial Series 09 | Julia Probabilistic Programming for Beginners is a wonderful, fairly recent entry this year by â€śdoggo dot jlâ€ť. A video series with accompanying repository of notebooks, this module follows some of McElreathâ€™s excellent, Statistical Rethinking book/lectures, but is an even more succinct and approachable (gentle) introduction, and is implemented in Julia/Turing.jl. Really enjoyable. Then you can follow this with some of the other resources mentioned above. (Actually, I strongly recommend anything by doggo dot.jl. Created with the wide-eyed, basic perspective of an inquisitive amateur, they efficiently explore many powerful aspects of the â€śvast julia wilderness,â€ť in a fun, thoughtful manner.) Good luck!

1 Like