# Multivariate dirichlet mixture with Turing

**URL:** <https://discourse.julialang.org/t/multivariate-dirichlet-mixture-with-turing/39546>\
**Category:** Probabilistic Programming\
**Tags:** turing\
**Created:** [May 15, 2020, 6:21pm UTC](https://discourse.julialang.org/t/multivariate-dirichlet-mixture-with-turing/39546 "2020-05-15T18:21:08Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![fooman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/fooman/32/14883_2.png) [@fooman](https://discourse.julialang.org/u/fooman)\
**Post date:** [May 15, 2020, 6:21pm UTC](https://discourse.julialang.org/t/multivariate-dirichlet-mixture-with-turing/39546/1 "2020-05-15T18:21:08Z")

</div>

Hi – I’m trying to fit a mixture of 4 bivariate gaussians with Turing and getting a strange error? Can someone help with the syntax here:

@model GaussianMixtureModel(x, K) = begin  
# K is expected number of clusters  
# x is bivariate data, typically 2 x 1000

```
D, N = size(x)

# Draw the paramters
μ ~ [MvNormal([0.,0.], 5.) for i in 1:K]
τ ~ [Gamma() for i in 1:K]

# Dirichlet prior
α = 1.0
w ~ Dirichlet(K, α)

# Draw assignments for each datum and generate it from a multivariate normal.
k = Vector{Int}(undef, N)
for i in 1:N
    k[i] ~ Categorical(w)
    x[:, i] ~ MvNormal(μ[k[i]], τ[k[i]])
end
    
return k

```

end

I’m getting error (when sampling): MethodError: no method matching assume(::DynamicPPL.SampleFromPrior,

Thanks

---

<div class="post-metadata">

**Author:** ![mohamed82008](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mohamed82008/32/18171_2.png) [@mohamed82008](https://discourse.julialang.org/u/mohamed82008)\
**Post date:** [May 15, 2020, 10:38pm UTC](https://discourse.julialang.org/t/multivariate-dirichlet-mixture-with-turing/39546/2 "2020-05-15T22:38:05Z")

</div>

This should work.

```julia
@model GaussianMixtureModel(x, K) = begin
    # K is expected number of clusters
    # x is bivariate data, typically 2 x 1000

    D, N = size(x)

    # Draw the paramters
    μ ~ filldist(MvNormal([0.,0.], 5.), K)
    τ ~ filldist(Gamma(), K)

    # Dirichlet prior
    α = 1.0
    w ~ Dirichlet(K, α)

    # Draw assignments for each datum and generate it from a multivariate normal.
    k = Vector{Int}(undef, N)
    for i in 1:N
        k[i] ~ Categorical(w)
        x[:, i] ~ MvNormal(μ[:, k[i]], τ[k[i]])
    end

    return k
end

```

---

<div class="post-metadata">

**Author:** ![fooman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/fooman/32/14883_2.png) [@fooman](https://discourse.julialang.org/u/fooman)\
**Post date:** [May 15, 2020, 11:02pm UTC](https://discourse.julialang.org/t/multivariate-dirichlet-mixture-with-turing/39546/3 "2020-05-15T23:02:50Z")

</div>

Awesome - this works! Thanks

---

<div class="post-metadata">

**Author:** ![fooman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/fooman/32/14883_2.png) [@fooman](https://discourse.julialang.org/u/fooman)\
**Post date:** [May 15, 2020, 11:59pm UTC](https://discourse.julialang.org/t/multivariate-dirichlet-mixture-with-turing/39546/4 "2020-05-15T23:59:49Z")

</div>

Are there any issues sampling from MvNormal() as used in the model above?

gmm\_model = GaussianMixtureModel(x, 8);  
num\_iter = 1000  
samp\_mh = sample(gmm\_model, MH(), num\_iter);  
samp\_hmc = sample(gmm\_model, HMC(0.1, 5), num\_iter);  
samp\_nuts = sample(gmm\_model, NUTS(100, 0.65), num\_iter);

If I try these three samplers MH() runs ok and completes. However both NUTS() and HMC() complain with the same error message: **“TypeError: in typeassert, expected Int64, got ForwardDiff.Dual{Nothing,Int64,10}”**

Any ideas? The synthetic dataset I’m trying to run this on is generated as follows:

y1 = rand(MvNormal([0., 0.], [[1, 0.] [0., 0.02]]), 250);  
y2 = rand(MvNormal([0., 0.], [[0.02, 0.] [0., 1]]), 400);  
y3 = rand(MvNormal([2., 2.], [[1., -0.9] [-0.9, 1]]), 500);  
y4 = rand(MvNormal([-2., -2.], [[0.1, 0.] [0., 0.1]]), 300);  
x = reduce(hcat, [y1, y2, y3, y4])

Thanks

---

<div class="post-metadata">

**Author:** ![mohamed82008](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mohamed82008/32/18171_2.png) [@mohamed82008](https://discourse.julialang.org/u/mohamed82008)\
**Post date:** [May 16, 2020, 12:05am UTC](https://discourse.julialang.org/t/multivariate-dirichlet-mixture-with-turing/39546/5 "2020-05-16T00:05:27Z")

</div>

You can’t use pure HMC when you have integer random variables, `k` above. You need `Gibbs` for that.

---

<div class="post-metadata">

**Author:** ![fooman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/fooman/32/14883_2.png) [@fooman](https://discourse.julialang.org/u/fooman)\
**Post date:** [May 16, 2020, 2:28pm UTC](https://discourse.julialang.org/t/multivariate-dirichlet-mixture-with-turing/39546/6 "2020-05-16T14:28:46Z")

</div>

Does filldist() work as well for InverseWishart()? I’m having some issues setting up covariance prior as:

`τ ~ filldist(InverseWishart(2+1, Matrix{Float64}([[3,0.5] [0.5,3]])), K)`

and getting error message: “MethodError: no method matching filldist(::InverseWishart{Float64,PDMats.PDMat{Float64,Array{Float64,2}}}, ::Int64)”

Here’s the full model:

```julia
@model GaussianMixtureModel(x, K) = begin
    # K is expected number of clusters
    # x is bivariate data, typically 2 x 1000
    
    D, N = size(x)
    
     # Draw the paramters
    μ ~ filldist(MvNormal([0.,0.], 5.), K)
    τ ~ filldist(InverseWishart(2+1, Matrix{Float64}([[3,0.5] [0.5,3]])), K)
    
    # Dirichlet prior
    α = 1.0
    w ~ Dirichlet(K, α)
    
    # Draw assignments for each datum and generate it from a multivariate normal.
    k = Vector{Int}(undef, N)
    for i in 1:N
        k[i] ~ Categorical(w)
        x[:, i] ~ MvNormal(μ[:, k[i]], τ[:, :, k[i]])
    end
        
    return k
end

```

Thanks

---

<div class="post-metadata">

**Author:** ![mohamed82008](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mohamed82008/32/18171_2.png) [@mohamed82008](https://discourse.julialang.org/u/mohamed82008)\
**Post date:** [May 16, 2020, 3:06pm UTC](https://discourse.julialang.org/t/multivariate-dirichlet-mixture-with-turing/39546/7 "2020-05-16T15:06:18Z")

</div>

Please open an issue in DistributionsAD.jl.