# How to do non-centered parameterization in Turing

**URL:** <https://discourse.julialang.org/t/how-to-do-non-centered-parameterization-in-turing/44702>\
**Category:** Statistics\
**Tags:** turing\
**Created:** [August 10, 2020, 5:27pm UTC](https://discourse.julialang.org/t/how-to-do-non-centered-parameterization-in-turing/44702 "2020-08-10T17:27:08Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![vkv](https://avatars.discourse-cdn.com/v4/letter/v/d9b06d/32.png) [@vkv](https://discourse.julialang.org/u/vkv)\
**Post date:** [August 10, 2020, 5:27pm UTC](https://discourse.julialang.org/t/how-to-do-non-centered-parameterization-in-turing/44702/1 "2020-08-10T17:27:08Z")

</div>

k is a random variable `k ~ Normal(μ, σ)`, μ and σ have some other priors. I want to use a non-centered parameterization of the prior. I want to do something like this

```julia
k_ ~ Normal(0, 1)
k = k_ * σ + μ

```

Since the above doesn’t use `~` Turing doesn’t record `k`, it only records `k_`. Is there a way to do this in Turing?

---

<div class="post-metadata">

**Author:** ![trappmartin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/trappmartin/32/1165_2.png) [@trappmartin](https://discourse.julialang.org/u/trappmartin)\
**Post date:** [August 12, 2020, 8:03am UTC](https://discourse.julialang.org/t/how-to-do-non-centered-parameterization-in-turing/44702/2 "2020-08-12T08:03:50Z")

</div>

We currently don’t have a good way to record the transformed variable, but you can apply the transformation after the inference. We will likely add a mechanism to track arbitrary variables soon.

---

<div class="post-metadata">

**Author:** ![vkv](https://avatars.discourse-cdn.com/v4/letter/v/d9b06d/32.png) [@vkv](https://discourse.julialang.org/u/vkv)\
**Post date:** [August 12, 2020, 8:14am UTC](https://discourse.julialang.org/t/how-to-do-non-centered-parameterization-in-turing/44702/3 "2020-08-12T08:14:00Z")

</div>

I’ve few extra questions

1. Can you please tell me if the logpdf of these distributions are implemented in a centered or non centered way? If they are implemented in a centered way then I will try to implement a new distribution with non centered parameterization.

2. What happens when I run `x ~ Normal()` in Turing. I think it would internally call the `logpdf(Normal(), x)`.

---

<div class="post-metadata">

**Author:** ![trappmartin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/trappmartin/32/1165_2.png) [@trappmartin](https://discourse.julialang.org/u/trappmartin)\
**Post date:** [August 14, 2020, 7:10am UTC](https://discourse.julialang.org/t/how-to-do-non-centered-parameterization-in-turing/44702/4 "2020-08-14T07:10:27Z")

</div>

> [@vkv](#):
>
> What happens when I run `x ~ Normal()` in Turing. I think it would internally call the `logpdf(Normal(), x)` .

Yes that is more or less what is happening if you run HMC or any other gradien-based inference algorithm. What exactly happens depends on how the respective sampler overloads the function call, but for HMC this is simply the evaluation of the logpdf.

---

<div class="post-metadata">

**Author:** ![trappmartin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/trappmartin/32/1165_2.png) [@trappmartin](https://discourse.julialang.org/u/trappmartin)\
**Post date:** [August 14, 2020, 7:14am UTC](https://discourse.julialang.org/t/how-to-do-non-centered-parameterization-in-turing/44702/5 "2020-08-14T07:14:09Z")

</div>

> [@vkv](#):
>
> Can you please tell me if the logpdf of these distributions are implemented in a centered or non centered way? If they are implemented in a centered way then I will try to implement a new distribution with non centered parameterization.

We currently don’t do any automatic reparamterisation of models. This is also quite a difficult problem in general. However, as in Stan you can simply write the model using a centered or a non-centered parameterisation. The only difficulty is that we currently don’t have a good way to retrieve transformed variables after the sampling, but this will likely change soonish as we have discussed solutions to this already.

---

<div class="post-metadata">

**Author:** ![vkv](https://avatars.discourse-cdn.com/v4/letter/v/d9b06d/32.png) [@vkv](https://discourse.julialang.org/u/vkv)\
**Post date:** [August 14, 2020, 7:46am UTC](https://discourse.julialang.org/t/how-to-do-non-centered-parameterization-in-turing/44702/6 "2020-08-14T07:46:32Z")

</div>

> [@trappmartin](#):
>
> We currently don’t do any automatic reparamterisation of models. This is also quite a difficult problem in general

Thank you for clarifying this.

* * *

I was wondering if Turing doesn’t need reparameterization. I’ve come across the Neal’s funnel and I’ve implemented it in Turing. The posterior plot shows that there’s a need to reparameterize to efficiently explore the posterior.

Centered:  
 ![cp](https://global.discourse-cdn.com/julialang/original/3X/4/c/4c6ea3899449539407f1a42c060712dc5cee4ef3.png)

Non-centered:  
 ![ncp](https://global.discourse-cdn.com/julialang/original/3X/7/7/7765219a7dfd2b69a6ffcb6f13982c17d5c811a4.png)

I’ve used the stan’s implementation of the funnel in Turing as shown [here](https://mc-stan.org/docs/2_18/stan-users-guide/reparameterization-section.html)

---

<div class="post-metadata">

**Author:** ![trappmartin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/trappmartin/32/1165_2.png) [@trappmartin](https://discourse.julialang.org/u/trappmartin)\
**Post date:** [August 14, 2020, 4:33pm UTC](https://discourse.julialang.org/t/how-to-do-non-centered-parameterization-in-turing/44702/7 "2020-08-14T16:33:36Z")

</div>

Really nice. We should also add this to the Turing docs.

---

<div class="post-metadata">

**Author:** ![vkv](https://avatars.discourse-cdn.com/v4/letter/v/d9b06d/32.png) [@vkv](https://discourse.julialang.org/u/vkv)\
**Post date:** [August 14, 2020, 4:51pm UTC](https://discourse.julialang.org/t/how-to-do-non-centered-parameterization-in-turing/44702/8 "2020-08-14T16:51:04Z")

</div>

Can I write this?

---

<div class="post-metadata">

**Author:** ![trappmartin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/trappmartin/32/1165_2.png) [@trappmartin](https://discourse.julialang.org/u/trappmartin)\
**Post date:** [August 14, 2020, 7:47pm UTC](https://discourse.julialang.org/t/how-to-do-non-centered-parameterization-in-turing/44702/9 "2020-08-14T19:47:28Z")

</div>

Uh that would be awesome!  
cc: @cpfiffer

---

<div class="post-metadata">

**Author:** ![cpfiffer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cpfiffer/32/208747_2.png) [@cpfiffer](https://discourse.julialang.org/u/cpfiffer)\
**Post date:** [August 16, 2020, 1:47am UTC](https://discourse.julialang.org/t/how-to-do-non-centered-parameterization-in-turing/44702/10 "2020-08-16T01:47:50Z")

</div>

Yeah, I’d be delighted to see this added!

---

<div class="post-metadata">

**Author:** ![trappmartin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/trappmartin/32/1165_2.png) [@trappmartin](https://discourse.julialang.org/u/trappmartin)\
**Post date:** [August 16, 2020, 2:31pm UTC](https://discourse.julialang.org/t/how-to-do-non-centered-parameterization-in-turing/44702/11 "2020-08-16T14:31:36Z")

</div>

Let us know if you need help or have any questions.

---

<div class="post-metadata">

**Author:** ![vkv](https://avatars.discourse-cdn.com/v4/letter/v/d9b06d/32.png) [@vkv](https://discourse.julialang.org/u/vkv)\
**Post date:** [August 17, 2020, 7:22am UTC](https://discourse.julialang.org/t/how-to-do-non-centered-parameterization-in-turing/44702/12 "2020-08-17T07:22:45Z")

</div>

Is there a way to reparameterize `MixtureModel`?

For example `MixtureModel([Uniform(0, 25), Normal(30,5)],[1/2, 1/2])`

I have found [this example](https://discourse.mc-stan.org/t/how-to-reparameterize-a-gaussian-mixture-model/12792/12) on stan discourse. I guess writing a full log posterior is one way to do this.

---

<div class="post-metadata">

**Author:** ![trappmartin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/trappmartin/32/1165_2.png) [@trappmartin](https://discourse.julialang.org/u/trappmartin)\
**Post date:** [September 3, 2020, 7:54am UTC](https://discourse.julialang.org/t/how-to-do-non-centered-parameterization-in-turing/44702/13 "2020-09-03T07:54:47Z")

</div>

Yes, you would have to increment the target value as in the STAN code.  
In Turing you can do this as for example as follows:

```julia
@model ...
  ... # priors
  lp = log( ... ) # log pdf of observation model
  Turing.acclogp!(_varinfo, lp) # increment internal target value
end

```

---

<div class="post-metadata">

**Author:** ![erman\_test](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/erman_test/32/29306_2.png) [@erman\_test](https://discourse.julialang.org/u/erman_test)\
**Post date:** [September 19, 2021, 9:38am UTC](https://discourse.julialang.org/t/how-to-do-non-centered-parameterization-in-turing/44702/14 "2021-09-19T09:38:50Z")

</div>

Is this possible now? I was just looking for the same thing. 🙂  
I don’t think I found it in the turing documentation