# Constrained parameters in Turing

**URL:** <https://discourse.julialang.org/t/constrained-parameters-in-turing/64086>\
**Category:** Probabilistic Programming\
**Tags:** question, turing\
**Created:** [July 5, 2021, 4:03pm UTC](https://discourse.julialang.org/t/constrained-parameters-in-turing/64086 "2021-07-05T16:03:42Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Elmo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/elmo/32/17979_2.png) [@Elmo](https://discourse.julialang.org/u/Elmo)\
**Post date:** [July 5, 2021, 4:03pm UTC](https://discourse.julialang.org/t/constrained-parameters-in-turing/64086/1 "2021-07-05T16:03:42Z")

</div>

Hi all,

Is it possible to set up a model in Turing so that the posterior distribution of some parameters satisfy a set of constraints?

For example, suppose I want to do linear regression, but from physical principles I know that the coefficient matrix (`L`) of my model must be positive definite, and the cross diagonal terms must be equal. E.g.,

```julia
Y ~ L * X # linear model of my process

# with variables I observed
Y = [y1, y2] 
X = [x1, x2]
# and parameters I want to infer
L = [L11 L12;
     L21 L22]

# subject to these constraints:
# L must be positive definite
L11 * L22 - L12 * L21 > 0 # Sylvester's criterion
# The cross diagonal terms of L must be equal
L21 == L21

```

Is it possible to ensure that the posterior distribution of `L` obey these constraints? I am also not sure what effect the prior has on the posterior, i.e. should the prior of `L` also satisfy the constraints?

---

<div class="post-metadata">

**Author:** ![Storopoli](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/storopoli/32/209278_2.png) [@Storopoli](https://discourse.julialang.org/u/Storopoli)\
**Post date:** [July 6, 2021, 7:30am UTC](https://discourse.julialang.org/t/constrained-parameters-in-turing/64086/2 "2021-07-06T07:30:38Z")

</div>

Looks pretty much like a Cholesky factorization you’ll need.

Take a look here: [DistributionsAD.jl/common.jl at fe2070012167b78c84a49733a0a64997e9533812 · TuringLang/DistributionsAD.jl · GitHub](https://github.com/TuringLang/DistributionsAD.jl/blob/fe2070012167b78c84a49733a0a64997e9533812/src/common.jl#L3-L29)

---

<div class="post-metadata">

**Author:** ![torfjelde](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/torfjelde/32/206542_2.png) [@torfjelde](https://discourse.julialang.org/u/torfjelde)\
**Post date:** [July 6, 2021, 10:41am UTC](https://discourse.julialang.org/t/constrained-parameters-in-turing/64086/3 "2021-07-06T10:41:03Z")

</div>

> Is it possible to ensure that the posterior distribution of `L` obey these constraints?

Yep; this can be done for arbitrary constraints by just adding `-Inf` to the logjoint, i.e. putting

```julia
if ... # you're check as to whether the constraints are valid
    Turing.@addlogprob! -Inf
    # Do an early return since this entire sample will
    # be rejected anyways.
    return nothing
end

```

in your model. _But_ this is a bad idea in a lot of cases, e.g. if you’re working with continuous variables and have equality constraints (like you’re wanting to do) since these will be satisfied with probability 0 😕

> I am also not sure what effect the prior has on the posterior, i.e. should the prior of `L` also satisfy the constraints?

Exactly. Whenever you can encode the constraints/symmetrics into your prior, do it:) In the particular example you mention you can for example drop the equality-constraint, and instead construct your `L` by using `L12` in both the `(1, 2)` and the `(2, 1)` entry.

If indeed the example you posted is _exactly_ what you want to do, i.e. you want a prior on a positive-definite matrix, then you always have the [`LKJ` distribution](https://juliastats.org/Distributions.jl/stable/matrix/#Distributions.LKJ) + sampling the diagonal-terms separately, or even [`Wishart`](https://juliastats.org/Distributions.jl/stable/matrix/#Distributions.Wishart) with parameters ensuring full-rank.

---

<div class="post-metadata">

**Author:** ![Elmo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/elmo/32/17979_2.png) [@Elmo](https://discourse.julialang.org/u/Elmo)\
**Post date:** [July 8, 2021, 7:40pm UTC](https://discourse.julialang.org/t/constrained-parameters-in-turing/64086/4 "2021-07-08T19:40:55Z")

</div>

In case anyone stumbles across this post, this worked for me:

```julia
@model function fit_onsager(x, y)
    σ ~ truncated(Normal(0, 100), 0, Inf)
    L ~ InverseWishart(2, Matrix{Float64}(I, 2, 2))

    for i=1:size(x, 2)
        y[:, i] ~ MvNormal(L * x[:, i], σ)
    end
end

model = fit_onsager(X, Y)
chain = sample(model, NUTS(0.65), 3_000);

```

In effect, the inverse Wishart distribution satisfies all my constraints 🙂 Note, the sampler sometimes fails to start, complaining about a PosDefException or a SingularException, but if you just restart it a few times it eventually works…