# Turing's summary statistics - ESS

**URL:** <https://discourse.julialang.org/t/turings-summary-statistics-ess/103537>\
**Category:** Statistics\
**Tags:** question, turing\
**Created:** [September 5, 2023, 9:18am UTC](https://discourse.julialang.org/t/turings-summary-statistics-ess/103537 "2023-09-05T09:18:12Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![PrimulaBunce](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/primulabunce/32/46933_2.png) [@PrimulaBunce](https://discourse.julialang.org/u/PrimulaBunce)\
**Post date:** [September 5, 2023, 9:18am UTC](https://discourse.julialang.org/t/turings-summary-statistics-ess/103537/1 "2023-09-05T09:18:12Z")

</div>

Hi,

I’m new to MCMC, and I’m using the Turing package to implement NUTS.

```julia
par_NUTS = sample(model, NUTS(1000, 0.65), MCMCThreads(), 1000, 3)

```

I’m trying to figure out what the Effective Sample Size (ESS) values returned in the summary statistics mean.

I’ve run 3 separate chains. When I check the ESS values for the individual chains and the combined chains they turn out to be identical:

```julia
summarystats(par_NUTS; append_chains=true) # Appended chains

```

![image](https://global.discourse-cdn.com/julialang/original/3X/3/d/3ddd2a73cc0c07eb82d6021d7ac3a156cc7f900b.png)

```julia
summarystats(par_NUTS; append_chains=false)[1] # Individual chains

```

![image](https://global.discourse-cdn.com/julialang/original/3X/d/b/dbc80720e66c51646f7dd14be96d9a5ffb8d19c0.png)

```julia
summarystats(par_NUTS; append_chains=false)[2] # Individual chains

```

![image](https://global.discourse-cdn.com/julialang/original/3X/7/6/76df336e6bb02c6d6652c1b88ddf028ffad8fac5.png)

```julia
summarystats(par_NUTS; append_chains=false)[3] # Individual chains

```

![image](https://global.discourse-cdn.com/julialang/original/3X/e/1/e1b0d0d8be5e4a2608bacc8fe140b3343b49e1ae.png)

What does this mean? Is it the relative ESS values that are returned?

---

<div class="post-metadata">

**Author:** ![skleinbo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/skleinbo/32/36080_2.png) [@skleinbo](https://discourse.julialang.org/u/skleinbo)\
**Post date:** [September 8, 2023, 10:16am UTC](https://discourse.julialang.org/t/turings-summary-statistics-ess/103537/2 "2023-09-08T10:16:26Z")

</div>

The documentation of `MCMCChains.rhat` references the paper [https://arxiv.org/pdf/1903.08008.pdf](https://arxiv.org/pdf/1903.08008.pdf)  
from which it apparently takes the calculation of `rhat` (and `ess`?).  
I have only skimmed over the first two pages just now, and hope I do understand correctly. The authors make the point that in general the convergence of MCMC cannot be reliably assessed from a single chain.

Thus, those observables are not calculated for every chain separately, which explains the identical values you observe. You may force a split by calling `ess_rhat(chains[:,:,1])` etc., but again, those might not reliably report failed convergence and overestimate the ESS.

(_P.S.: Please do not double post ['ESS' in Turing.jl](https://discourse.julialang.org/t/ess-in-turing-jl/103650) if a question doesn’t receive any attention for a couple of days. That happens. Instead, you can bump it with a comment after a while._)

---

<div class="post-metadata">

**Author:** ![sethaxen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sethaxen/32/35604_2.png) [@sethaxen](https://discourse.julialang.org/u/sethaxen)\
**Post date:** [October 6, 2023, 7:35am UTC](https://discourse.julialang.org/t/turings-summary-statistics-ess/103537/3 "2023-10-06T07:35:31Z")

</div>

I think this is worth a bug report in MCMCChains. While an argument can be made that the R-hat computed for all chains should be returned for each chain, the same cannot be said for the ESS. It’s absolutely incorrect to return for a single chain the ESS or MCSE for all chains together. It’s probably best that MCMCChains then computes the ESS, R-hat, and MCSE for each chain separately. While this decreases the usefulness of the diagnostics, it avoids potential footguns.