# How to integrate out latent variables/ intermediate values in Turing model

**URL:** <https://discourse.julialang.org/t/how-to-integrate-out-latent-variables-intermediate-values-in-turing-model/89003>\
**Category:** Modelling & Simulations\
**Tags:** question, turing, bayesian-inference\
**Created:** [October 20, 2022, 10:59am UTC](https://discourse.julialang.org/t/how-to-integrate-out-latent-variables-intermediate-values-in-turing-model/89003 "2022-10-20T10:59:41Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![neuro\_enthusiast](https://avatars.discourse-cdn.com/v4/letter/n/43a26b/32.png) [@neuro\_enthusiast](https://discourse.julialang.org/u/neuro_enthusiast)\
**Post date:** [October 20, 2022, 10:59am UTC](https://discourse.julialang.org/t/how-to-integrate-out-latent-variables-intermediate-values-in-turing-model/89003/1 "2022-10-20T10:59:41Z")

</div>

Hi all, my generative model has the following form.  
 P(y |\theta) = \int P(y|x) P(x|\theta) dx   
I want to get the posterior of θ given fixed data y. But I unfortunately don’t have an analytical solution for the integral over x. As a result, I’m implementing the model in turing with the following structure and using sampling to get a posterior on θ

```julia
@model my_func()
   θ ~ prior_dist()
   x ~ dist1(θ)
   y ~ dist2(x)
end

```

However, this results in a really high number of dimensions for sampling, since all the x are also in the posterior. Is there a way to integrate out a latent variable in Turing to avoid this problem?

---

<div class="post-metadata">

**Author:** ![dlakelan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlakelan/32/8491_2.png) [@dlakelan](https://discourse.julialang.org/u/dlakelan)\
**Post date:** [October 20, 2022, 1:13pm UTC](https://discourse.julialang.org/t/how-to-integrate-out-latent-variables-intermediate-values-in-turing-model/89003/2 "2022-10-20T13:13:10Z")

</div>

dist1 is a high dimensional multivariate dist?

Unless you have an analytic form there isn’t going to be an easy way to do high dim integration

---

<div class="post-metadata">

**Author:** ![neuro\_enthusiast](https://avatars.discourse-cdn.com/v4/letter/n/43a26b/32.png) [@neuro\_enthusiast](https://discourse.julialang.org/u/neuro_enthusiast)\
**Post date:** [October 20, 2022, 1:40pm UTC](https://discourse.julialang.org/t/how-to-integrate-out-latent-variables-intermediate-values-in-turing-model/89003/3 "2022-10-20T13:40:45Z")

</div>

Indeed dist1 is a high dimensional multivariate dist. In my case, it’s something along the lines of MvNormal(0, θ). However, there are some transformations in between in my actual model so I couldn’t integrate analytically.

Is it perhaps possible to marginalize out x by sampling x within each turing sampling step, assuming that the following is correct?  
P(y|\theta) = \int P(y|x)P(x|\theta) dx = \frac{1}{N} \sum\_{i=1}^{N} P(y|x\_i)  
x \sim dist1(x|\theta)

---

<div class="post-metadata">

**Author:** ![dlakelan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlakelan/32/8491_2.png) [@dlakelan](https://discourse.julialang.org/u/dlakelan)\
**Post date:** [October 20, 2022, 2:12pm UTC](https://discourse.julialang.org/t/how-to-integrate-out-latent-variables-intermediate-values-in-turing-model/89003/4 "2022-10-20T14:12:58Z")

</div>

I think you’re trying to do something like:

```julia
theta ~ MyPrior()
x = rand(MvNormal(0,theta))
xprime = transform(x)
inclp = sum(logpdf(Ydist(xp),y) for xp in xprime)
@addlogprob!(inclp)

```

Which you can definitely try… see what you think.

---

<div class="post-metadata">

**Author:** ![neuro\_enthusiast](https://avatars.discourse-cdn.com/v4/letter/n/43a26b/32.png) [@neuro\_enthusiast](https://discourse.julialang.org/u/neuro_enthusiast)\
**Post date:** [October 21, 2022, 9:27am UTC](https://discourse.julialang.org/t/how-to-integrate-out-latent-variables-intermediate-values-in-turing-model/89003/5 "2022-10-21T09:27:05Z")

</div>

Thanks so much for this solution! I was just introduced to @addlogprob! the other day but didn’t realize you can use it like this.