Using Turing with ad-hoc likelihood

I am new to probabilistic programming an I would like to use Turing to perform statistical inference for a relatively complex physical problem, involving a specific model and a set of observations. The model has a few nuisance parameters: that is, the likelihood can be written as P(D \mid \theta, \eta), where D are the data (observations), \theta are the “interesting” parameters and \eta the “nuisance” parameters.

When performing the Bayesian inference I can obtain the posterior P(\theta, \eta \mid D), but given that I am not interested in the nuisance parameters \eta I would anyway marginalize the posterior over them. This is certainly possible, and I believe I know how I could do it with Turing.

Now, the specific form of the likelihood and the assumed prior for \eta let me simplify the problem: I can actually write directly a marginalized likelihood P(D \mid \theta), where I have de-fact removed the nuisance parameters.

The problem is now that I know how to compute P(D \mid \theta), but I do not know anymore how to formulate the problem in Turing: after the marginalization, this function cannot be written anymore in a simple way (for example, it is not anymore the product of the individual likelihoods of each datum). I tried some unsuccessful experiments using directly Turing.@addlogprob!, but I understand this is an advanced topic and there are a number of issues to take into account (including the sampler), so probably this is not the way to go.

2 Likes

One option is to implement the marginal distribution P(D | θ) directly as a Distribution (in the Distributions.jl) sense, and put that in your programme. This is essentially what we do with GPs from the JuliaGPs ecosystem.

Some of the main Turing devs will likely know if there’s a different option.

1 Like