# Turing: problem using MCMCthreads()

**URL:** <https://discourse.julialang.org/t/turing-problem-using-mcmcthreads/78906>\
**Category:** Statistics\
**Tags:** turing\
**Created:** [April 2, 2022, 8:44am UTC](https://discourse.julialang.org/t/turing-problem-using-mcmcthreads/78906 "2022-04-02T08:44:01Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![mcreel](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mcreel/32/30088_2.png) [@mcreel](https://discourse.julialang.org/u/mcreel)\
**Post date:** [April 2, 2022, 8:44am UTC](https://discourse.julialang.org/t/turing-problem-using-mcmcthreads/78906/1 "2022-04-02T08:44:01Z")

</div>

I am getting an error when trying to use MCMCthreads() for sampling. The following code works fine:

```julia
chain = sample(MSM(m, S, model),
    MH(:θt => AdvancedMH.RandomWalkProposal(MvNormal(zeros(size(m,1)), tuning*Σp))),
    length; init_params=m, discard_initial=burnin)

```

However, the following code

```julia
chain = sample(MSM(m, S, model),
    MH(:θt => AdvancedMH.RandomWalkProposal(MvNormal(zeros(size(m,1)), tuning*Σp))),
    MCMCThreads(), length, nchains; init_params=m, discard_initial=burnin)

```

gives an error “Provided initial value doesn’t match the dimension of the model”.

Before either of these are called, nchains, length, burnin and m are all defined, and are the same for both of the above calls. Also, Julia is started with julia -t 4, and the environment variable JULIA\_NUM\_THREADS=4 is set.

The code is from the file [https://github.com/mcreel/SimulatedNeuralMoments.jl/blob/main/examples/MN/MNexample.jl](https://github.com/mcreel/SimulatedNeuralMoments.jl/blob/main/examples/MN/MNexample.jl)

Strangely enough, the file [https://github.com/mcreel/SimulatedNeuralMoments.jl/blob/main/examples/SV/SVexample.jl](https://github.com/mcreel/SimulatedNeuralMoments.jl/blob/main/examples/SV/SVexample.jl)

contains identical code:

```julia
chain = sample(MSM(m, S, model),
    MH(:θt => AdvancedMH.RandomWalkProposal(MvNormal(zeros(size(m,1)), tuning*Σp))),
    MCMCThreads(), length, nchains; init_params=m, discard_initial=burnin)

# single thread
#=
chain = sample(MSM(m, S, model),
    MH(:θt => AdvancedMH.RandomWalkProposal(MvNormal(zeros(3), tuning*Σp))),
    length; init_params = m, discard_initial=burnin)
=#

```

and both of these run without problems. If anyone sees the problem, that would be much appreciated. If not, I guess I should file an issue with Turing.

---

<div class="post-metadata">

**Author:** ![Christopher\_Fisher](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/christopher_fisher/32/26132_2.png) [@Christopher\_Fisher](https://discourse.julialang.org/u/Christopher_Fisher)\
**Post date:** [April 2, 2022, 7:41pm UTC](https://discourse.julialang.org/t/turing-problem-using-mcmcthreads/78906/2 "2022-04-02T19:41:03Z")

</div>

I’m not quite sure what is wrong. Can you produce the behavior with a simpler example, such as estimating mu and sigma of a normal distribution? If so, it might be worth filing an issue.

---

<div class="post-metadata">

**Author:** ![mcreel](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mcreel/32/30088_2.png) [@mcreel](https://discourse.julialang.org/u/mcreel)\
**Post date:** [April 2, 2022, 8:09pm UTC](https://discourse.julialang.org/t/turing-problem-using-mcmcthreads/78906/3 "2022-04-02T20:09:13Z")

</div>

The odd thing is that the two files, MNexample.jl and SVexample.jl, are essentially identical from Turing’s point of view - they just implement two different statistical models. I will try to boil down the file that has the problem to a minimum working example.

---

<div class="post-metadata">

**Author:** ![mcreel](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mcreel/32/30088_2.png) [@mcreel](https://discourse.julialang.org/u/mcreel)\
**Post date:** [April 3, 2022, 8:05am UTC](https://discourse.julialang.org/t/turing-problem-using-mcmcthreads/78906/4 "2022-04-03T08:05:19Z")

</div>

Here is a MWE, or better said, M non-working E.

With the 5 or 4 dimensional problems, the error is

```julia
 
        nested task error: Provided initial value doesn't match the dimension of the model

```

With the 3 dimensional problem, the error is

```julia
ERROR: LoadError: ArgumentError: not enough initial parameters (expected 4, received 3

```

In all cases, julia was started with `julia -t 4 --proj` where the project has the needed packages. If the block using MCMCThreads() is commented out, the single thread version works in all cases.

```julia
using Turing, AdvancedMH, LinearAlgebra

#lb = [0.0, 0.0, 0.0, 0.0, 0.05]
#ub = [3.0, 3.0, 1.0, 3.0, 0.95] 
#m = [1.0, 1.0, 0.2, 1.8, 0.4] 

lb = [0.0, 0.0, 0.0, 0.0]
ub = [3.0, 3.0, 1.0, 3.0] 
m = [1.0, 1.0, 0.2, 1.8] 

#lb = [0.0, 0.0, 0.0]
#ub = [3.0, 3.0, 1.0] 
#m = [1.0, 1.0, 0.2] 

k = size(m,1)
V = 1.0* Matrix(I, k, k)

macro Prior()
    return :( arraydist([Uniform(lb[i], ub[i]) for i = 1:size(lb,1)]) )
end

transf = bijector(@Prior) # transforms draws from prior to draws from ℛⁿ 
transformed_prior = transformed(@Prior, transf) # the transformed prior

@model function MSM(m)
    θt ~ transformed_prior
    m ~ MvNormal(θt, V)
end

length = 1000
nchains = 4
# This does not work
chain = sample(MSM(m),
        MH(:θt => AdvancedMH.RandomWalkProposal(MvNormal(zeros(k), V))),
        MCMCThreads(), length, nchains;
        init_params=m)
#
# single thread, this works
chain = sample(MSM(m),
        MH(:θt => AdvancedMH.RandomWalkProposal(MvNormal(zeros(k), V))),
        length;
        init_params=m)

```

---

<div class="post-metadata">

**Author:** ![mcreel](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mcreel/32/30088_2.png) [@mcreel](https://discourse.julialang.org/u/mcreel)\
**Post date:** [April 3, 2022, 10:18am UTC](https://discourse.julialang.org/t/turing-problem-using-mcmcthreads/78906/5 "2022-04-03T10:18:54Z")

</div>

Update: the error is occurring with Turing v0.21.1, but it does not occur with v0.20.2. I will try to narrow down exactly which version provokes the error, and then I suppose I should file an issue with Turing.jl

UPDATE: it works with Turing 0.20.4, but not with 0.21.0.

in both cases, the other Turing related packages are at  
AdvancedMH 0.6.7  
MCMCChains 5.1.0

---

<div class="post-metadata">

**Author:** ![Christopher\_Fisher](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/christopher_fisher/32/26132_2.png) [@Christopher\_Fisher](https://discourse.julialang.org/u/Christopher_Fisher)\
**Post date:** [April 3, 2022, 11:01am UTC](https://discourse.julialang.org/t/turing-problem-using-mcmcthreads/78906/6 "2022-04-03T11:01:36Z")

</div>

I tend to agree. It sounds like there might be a bug, or an intended change in behavior. However, I did not see anything in the closed issues or merged pull requests that immediately suggest the later.

---

<div class="post-metadata">

**Author:** ![mcreel](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mcreel/32/30088_2.png) [@mcreel](https://discourse.julialang.org/u/mcreel)\
**Post date:** [April 3, 2022, 11:11am UTC](https://discourse.julialang.org/t/turing-problem-using-mcmcthreads/78906/7 "2022-04-03T11:11:53Z")

</div>

I have filed an issue, at [https://github.com/TuringLang/Turing.jl/issues/1810](https://github.com/TuringLang/Turing.jl/issues/1810)

The same problem occurs with NUTS, which is what is used in the example code in the issue.

---

<div class="post-metadata">

**Author:** ![sethaxen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sethaxen/32/35604_2.png) [@sethaxen](https://discourse.julialang.org/u/sethaxen)\
**Post date:** [April 3, 2022, 12:29pm UTC](https://discourse.julialang.org/t/turing-problem-using-mcmcthreads/78906/8 "2022-04-03T12:29:38Z")

</div>

From the AbstractMCMC docs:

> - `init_params` (default: `nothing`): if set to `init_params !== nothing`, then the `i`th element of `init_params` is used as initial parameters of the `i`th chain. If one wants to use the same initial parameters `x` for every chain, one can specify e.g. `init_params = Iterators.repeated(x)` or `init_params = FillArrays.Fill(x, N)`.

So when sampling a single chain, you pass a single set of initial parameters. When sampling multiple chains, you pass an iterator of initial parameters. e.g.

```julia
julia> using Turing

julia> @model function mod()
           x ~ filldist(Normal(), 10)
       end;

julia> x0 = randn(10);

julia> sample(mod(), NUTS(), 1_000; init_params=x0);
┌ Info: Found initial step size
└ ϵ = 1.6
Sampling 100%|█████████████████████████████████████████████████████████████████████████| Time: 0:00:00

julia> sample(mod(), NUTS(), MCMCThreads(), 1_000, 4; init_params=Iterators.repeated(x0));
┌ Warning: Only a single thread available: MCMC chains are not sampled in parallel
└ @ AbstractMCMC ~/.julia/packages/AbstractMCMC/0eT8o/src/sample.jl:291
┌ Info: Found initial step size
└ ϵ = 1.6
┌ Info: Found initial step size
└ ϵ = 1.6
┌ Info: Found initial step size
└ ϵ = 1.6
┌ Info: Found initial step size
└ ϵ = 1.6
Sampling (1 threads) 100%|█████████████████████████████████████████████████████████████| Time: 0:00:00

```

---

<div class="post-metadata">

**Author:** ![mcreel](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mcreel/32/30088_2.png) [@mcreel](https://discourse.julialang.org/u/mcreel)\
**Post date:** [April 3, 2022, 4:12pm UTC](https://discourse.julialang.org/t/turing-problem-using-mcmcthreads/78906/9 "2022-04-03T16:12:15Z")

</div>

Thanks, that solved it. I will close the issue I opened.