Hi All, This question might reveal my limited knowledge of Bayesian estimation, but I am having difficulty figuring out how to perform basic Bayesian model selection in Turing. The documentation for MCMCChains.jl has a short section on DIC, which I’m not very familiar with. How does one calculate B…

@ANasc , just a quick status update: DIC was added to a new repo called ModelComparisons, along with BIC and AIC. This repo also has WAIC and PSILOO. It will take a few days for registration to be approved. I will also submit a change to MCMCChains so that you can calculate DIC directly from a Chain …

[image] ANasc: How does one calculate Bayes factors, for example? I don’t know much about the metrics for model selection, but do know that various statisticians argue against Bayes factors (Tendeiro & Kiers, 2019 ; Gelman et al., 2020 ). As far as I understand, McElreath ( 2020 ) and Gelman et …

As far as I can tell, there are two options currently, First, there is a Julia interface to ArviZ , which is written in Python. Second, there is a package associated with the book Statistical Rethinking that implements WAIC and LOO. The downside to the second option is that it does not currently have…

Thank you both for the links, these are very useful. I will brush up on which approaches are most appropriate for my case and see if I can implement something. I’ll also look more closely at DIC in MCMCChains. I can’t seem to get it to work. It gives me an error that I haven’t used the correct inpu…

[image] ANasc: lp += map(p -> logpdf( ... , x), Array(chain[:,i,:])) I think the problem is that you are supposed to add your distribution and data to the code. For example, lpfun = function f(chain::Chains) # function to compute the logpdf values niter, nparams, …

Yeah, I think that’s probably a better way to do it. It seems like a pretty trivial thing to add that extension into MCMCChains – maybe a good first issue for somebody.

I tried to change the API, but I encountered several issues with the lpfun function. It almost seems like it was never fully tested and implemented. I opened an issue .

@Christopher_Fisher Thanks for working on this, I really appreciate it.

`No problem. The new plan is to add DIC to PSS tomorrow or Saturday.

Amazing! Thanks so much. I look forward to trying it out. I think this will be a very useful addition for a lot of people.

Great Job @Christopher_Fisher ! Excited to see PSIS-LOO coming to life in MCMCChains.jl!

Bayesian Model Selection

Specific Domains Statistics

brett_knoss May 2, 2021, 8:58pm 19

Looks, good, is there a package version, or do I need to put it together. I take it I can use MCMC, and it looks like this uses Mann Whitney U Test?

Topic		Replies	Views
WAIC, or LOO etc in Turing? Probabilistic Programming	11	4628	October 1, 2020
Bayesian model comparison / selection Statistics question , turing , bayesian-inference	3	1292	May 3, 2022
Compute Deviance Information Criterion for a Bayesian Model Probabilistic Programming question , turing , mcmc	1	131	September 9, 2025
Bayes Testing Statistics question , package	27	2056	May 12, 2021
New Bayesian Statistics tutorials with Turing Probabilistic Programming turing	25	4853	July 15, 2022

Bayesian Model Selection

Related topics