# Within-chain parallelization with Turing.jl
**URL:** https://discourse.julialang.org/t/within-chain-parallelization-with-turing-jl/103402
**Category:** Probabilistic Programming
**Tags:** turing
**Created:** [August 31, 2023, 1:23pm UTC](https://discourse.julialang.org/t/within-chain-parallelization-with-turing-jl/103402 "2023-08-31T13:23:55Z")
**Posts on this page:** 7
**Page:** 1
### Author:  [@Nicolo\_Foppa](https://discourse.julialang.org/u/Nicolo_Foppa)
#### Post date: [August 31, 2023, 1:23pm UTC](https://discourse.julialang.org/t/within-chain-parallelization-with-turing-jl/103402/1 "2023-08-31T13:23:55Z")
Hi everyone,
I’m a new Turing user and I read that there is a possible way to parallelize the MCMC of a single chain on multiple threads but I’m not able to do that. Do you have any references that I can use to have a better understanding of that?
Thank you,
Nicolo’
---
### Author:  [@Red-Portal](https://discourse.julialang.org/u/Red-Portal)
#### Post date: [September 1, 2023, 6:21pm UTC](https://discourse.julialang.org/t/within-chain-parallelization-with-turing-jl/103402/2 "2023-09-01T18:21:49Z")
Could you upload your attempt so that we could take a look into what’s not working?
---
### Author:  [@dlakelan](https://discourse.julialang.org/u/dlakelan)
#### Post date: [September 1, 2023, 8:44pm UTC](https://discourse.julialang.org/t/within-chain-parallelization-with-turing-jl/103402/3 "2023-09-01T20:44:50Z")
> [@Nicolo\_Foppa](#):
>
> parallelize the MCMC of a single chain on multiple threads
You should be able to utilize whatever `@threads` and `@spawn` and other parallel features of Julia from within a model. There is no way I know of to automatically parallelize a chain in some way. It is possible to run several chains in parallel easily though.
---
### Author:  [@Nicolo\_Foppa](https://discourse.julialang.org/u/Nicolo_Foppa)
#### Post date: [September 1, 2023, 9:07pm UTC](https://discourse.julialang.org/t/within-chain-parallelization-with-turing-jl/103402/4 "2023-09-01T21:07:02Z")
I was trying to:
`sample(model, NUTS(), MCMCDistributed(), 50000, 1)`
using the `@everywhere` and I also tried:
`sample(model, NUTS(), MCMCThreads(), 50000, 1)`
None of the two approaches seemed working for the single chain while they are working well running multiple chains in parallel.
Thank you,
Nicolo’
---
### Author:  [@torfjelde](https://discourse.julialang.org/u/torfjelde)
#### Post date: [September 1, 2023, 9:31pm UTC](https://discourse.julialang.org/t/within-chain-parallelization-with-turing-jl/103402/5 "2023-09-01T21:31:29Z")
The follwing is probably more like what you’re referring to:
```julia
@model function demo_threading(x)
s ~ InverseGamma(2, 3)
m ~ Normal(0, sqrt(s))
# This will then be parallelized over the available threads.
Threads.@threads for i in eachindex(x)
x[i] ~ Normal(m, sqrt(s))
end
end
```
Note that this only works for observe-statements, i.e. when the LHS of `~` is “fixed” / not random (in the above example `x` is considered an observation because it’s part of the model arguments).
---
### Author:  [@Nicolo\_Foppa](https://discourse.julialang.org/u/Nicolo_Foppa)
#### Post date: [September 1, 2023, 9:38pm UTC](https://discourse.julialang.org/t/within-chain-parallelization-with-turing-jl/103402/6 "2023-09-01T21:38:46Z")
Yes! Thank you so much!
---
### Author:  [@torfjelde](https://discourse.julialang.org/u/torfjelde)
#### Post date: [September 1, 2023, 9:56pm UTC](https://discourse.julialang.org/t/within-chain-parallelization-with-turing-jl/103402/7 "2023-09-01T21:56:54Z")
Note that this is also mainly just compatible with ForwardDiff.jl in case you’re using samplers which require differentiation 😕 Unfortunately reverse-mode AD backends aren’t threadsafe (when used in this “naive” way).
Also, you can use [API · DynamicPPL](https://turinglang.org/library/DynamicPPL/stable/api/#DynamicPPL.@addlogprob)! to add log-prob computations performed by hand.