# MCMC Sampling Method Overview/Comparison?

**URL:** <https://discourse.julialang.org/t/mcmc-sampling-method-overview-comparison/88496>\
**Category:** Statistics\
**Created:** [October 9, 2022, 7:58pm UTC](https://discourse.julialang.org/t/mcmc-sampling-method-overview-comparison/88496 "2022-10-09T19:58:52Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Alec\_Loudenback](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/alec_loudenback/32/278_2.png) [@Alec\_Loudenback](https://discourse.julialang.org/u/Alec_Loudenback)\
**Post date:** [October 9, 2022, 7:58pm UTC](https://discourse.julialang.org/t/mcmc-sampling-method-overview-comparison/88496/1 "2022-10-09T19:58:52Z")

</div>

Is there a reference that compares the different sampling techniques available for Turing.jl? I don’t see a consistently preferred chain and it’s not clear to me (a beginner bayesian) why one would prefer a given sampler over another?

I found [this performance](https://dm13450.github.io/2019/04/10/Turing-Sampling-Speed.html) comparison - though three years ago is a long time ago for the ecosystem.

I have also read from various posts online that it’s difficult to determine a preferred sampler because which performs best (sampling accuracy, speed) will depend on problem specifics. Could folks share what some of the heurisitcs are for different samplers?

---

<div class="post-metadata">

**Author:** ![Alec\_Loudenback](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/alec_loudenback/32/278_2.png) [@Alec\_Loudenback](https://discourse.julialang.org/u/Alec_Loudenback)\
**Post date:** [October 11, 2022, 1:49am UTC](https://discourse.julialang.org/t/mcmc-sampling-method-overview-comparison/88496/2 "2022-10-11T01:49:27Z")

</div>

Another question: does Turing support sampling on GPU (e.g. CUDA)? I have played around with it but not been successful (e.g. sample below).

However, [posts like this one](https://discourse.julialang.org/t/gpu-support-for-turing-modeling-with-system-of-odes/77410/2) seem to indicate that it’s possible (and should just work out of the box?).

> **Attempted CUDA code**
>
> ```julia
> @model function model_gaussian2(claims,n)
> μ ~ Normal(0.05,0.1)
> σ ~ Exponential(0.25)
> 	
> claims .~ Binomial.(n,logistic.(μ))
> end
> 
> ```
> 
> where `claims` and `n` are `CuArrays` of integers:
> 
> ```julia
> mg = model_gaussian2(CuArray(claims_summary.claims),CuArray(claims_summary.n))
> cg = sample(mg, NUTS(), 500)
> 
> ```
> 
> Resutls in:
> 
> ```julia
> InvalidIRError: compiling kernel #broadcast_kernel#17(CUDA.CuKernelContext, CUDA.CuDeviceVector{Float64, 1}, Base.Broadcast.Broadcasted{CUDA.CuArrayStyle{1}, Tuple{Base.OneTo{Int64}}, typeof(StatsAPI.loglikelihood), Tuple{Base.Broadcast.Extruded{CUDA.CuDeviceVector{Distributions.Binomial{Float64}, 1}, Tuple{Bool}, Tuple{Int64}}, Base.Broadcast.Extruded{CUDA.CuDeviceVector{Int64, 1}, Tuple{Bool}, Tuple{Int64}}}}, Int64) resulted in invalid LLVM IR
> 
> Reason: unsupported call through a literal pointer (call to .text)
> 
> ```

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [October 11, 2022, 3:47am UTC](https://discourse.julialang.org/t/mcmc-sampling-method-overview-comparison/88496/3 "2022-10-11T03:47:53Z")

</div>

> [@Alec\_Loudenback](#):
>
> However, [posts like this one](https://discourse.julialang.org/t/gpu-support-for-turing-modeling-with-system-of-odes/77410/2) seem to indicate that it’s possible (and should just work out of the box?).

It depends on what and how you GPU. In that code, Turing would never see a GPU array, so as long as the code is ForwardDiff compatible it would work fine.

---

<div class="post-metadata">

**Author:** ![Alec\_Loudenback](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/alec_loudenback/32/278_2.png) [@Alec\_Loudenback](https://discourse.julialang.org/u/Alec_Loudenback)\
**Post date:** [October 11, 2022, 4:31am UTC](https://discourse.julialang.org/t/mcmc-sampling-method-overview-comparison/88496/4 "2022-10-11T04:31:30Z")

</div>

> [@ChrisRackauckas](#):
>
> Turing would never see a GPU array

I’m not sure I follow what you mean? The `@model` is given `CuArrays` for each of the arguments.

> [@ChrisRackauckas](#):
>
> So as long as the code is ForwardDiff compatible it would work fine.

“work fine” as in it would run without error on CPU, or it should have GPU-accelerated sampling?

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [October 11, 2022, 4:32am UTC](https://discourse.julialang.org/t/mcmc-sampling-method-overview-comparison/88496/5 "2022-10-11T04:32:51Z")

</div>

> [@Alec\_Loudenback](#):
>
> I’m not sure I follow what you mean? The `@model` is given `CuArrays` for each of the arguments.

No, not in the correct version of that code.

> [@Alec\_Loudenback](#):
>
> “work fine” as in it would run without error on CPU, or it should have GPU-accelerated sampling?

No error in the GPU-accelerated form.

---

<div class="post-metadata">

**Author:** ![Alec\_Loudenback](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/alec_loudenback/32/278_2.png) [@Alec\_Loudenback](https://discourse.julialang.org/u/Alec_Loudenback)\
**Post date:** [October 11, 2022, 1:23pm UTC](https://discourse.julialang.org/t/mcmc-sampling-method-overview-comparison/88496/6 "2022-10-11T13:23:26Z")

</div>

Thanks, for the clarification. Can you point me in the right direction with either an example online of how to do GPU-accelerated sampling or what I need to change in the code I have above?

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [October 13, 2022, 5:41am UTC](https://discourse.julialang.org/t/mcmc-sampling-method-overview-comparison/88496/7 "2022-10-13T05:41:48Z")

</div>

I’m not sure. That’s a very different type of GPU parallelism from the other example, and it would require deeper support from within Turing.

---

<div class="post-metadata">

**Author:** ![Red-Portal](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/red-portal/32/9102_2.png) [@Red-Portal](https://discourse.julialang.org/u/Red-Portal)\
**Post date:** [June 10, 2023, 5:50am UTC](https://discourse.julialang.org/t/mcmc-sampling-method-overview-comparison/88496/8 "2023-06-10T05:50:19Z")

</div>

Quite simple. NUTS almost always. If the prior is Gaussian and fairly low dimensional, elliptical slice sampling can be a good idea. If you’re damn sure that the posterior looks like a Gaussian and you have a huge dataset, maaaaybe SGLD.