# What is the interpretation of Turing's std, naive\_se, mcse?

**URL:** <https://discourse.julialang.org/t/what-is-the-interpretation-of-turings-std-naive-se-mcse/52252>\
**Category:** General Usage\
**Tags:** turing\
**Created:** [December 23, 2020, 2:02am UTC](https://discourse.julialang.org/t/what-is-the-interpretation-of-turings-std-naive-se-mcse/52252 "2020-12-23T02:02:46Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![jzr](https://avatars.discourse-cdn.com/v4/letter/j/eb9ed0/32.png) [@jzr](https://discourse.julialang.org/u/jzr)\
**Post date:** [December 23, 2020, 2:02am UTC](https://discourse.julialang.org/t/what-is-the-interpretation-of-turings-std-naive-se-mcse/52252/1 "2020-12-23T02:02:46Z")

</div>

An MCMC chain summary gives

```julia
Summary Statistics
  parameters mean std naive_se mcse ess rhat 
      Symbol Float64 Float64 Float64 Float64 Float64 Float64 

```

What is the difference between std, naive\_se, and mcse in terms of:

- how they are computed
- how they should be interpreted
- how they should be used/reported

Thanks.

---

<div class="post-metadata">

**Author:** ![rikh](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rikh/32/204104_2.png) [@rikh](https://discourse.julialang.org/u/rikh)\
**Post date:** [February 22, 2021, 9:36am UTC](https://discourse.julialang.org/t/what-is-the-interpretation-of-turings-std-naive-se-mcse/52252/2 "2021-02-22T09:36:06Z")

</div>

The Effective Sample Size (ESS) and Monte Carlo Standard Error (MCSE) are described in Kai Xu’s [thesis](https://www.mlmi.eng.cam.ac.uk/files/kai_xu_8224821_assignsubmission_file_xu_kai_dissertation.pdf) at page 16:

> **Effective Sample Size** ESS is a measure of how well a continuous chain is mixing. For a give chain \{x\_i\}\_{1:n}, ESS is defined by
> 
> \text{ESS} = \frac{n}{1 + \Sigma\_{k=1}^\infty \rho\_k},
> 
> where n is the total number of samples in the chain and \rho\_k is the autocorrelation factor at lag k of the chain [11].
> 
> An ESS measures how many samples are effective in the chain, the larger this value is, the higher the sampling efficiency. Different MCMC samplers can be evaluated by generating the same number of samples and comparing the ESS for each sampling results.
> 
> **Monte Carlo Standard Error** MCSE is an estimate of the inaccuracy of MC samples. There are multiple ways to estimate MCSE, among which the batch mean method proposed in [12] is believed to be the most popular one.
> 
> […]
> 
> As MCSE measures the inaccuracy of MC samples, a smaller value of MCSE is an indicator of better sampling performance.
> 
> However, it has been argued that MCSE is generally unimportant when the goal of inference is parameters themselves rather than the expectation of parameters, in which case the ESS is would be a more important measure [13].
> 
> ### References
> 
> [11] Dr. Orlaith Burke. _Statistical Methods Autocorrelation: MCMC Output Analysis._ Department of Statistics, University of Oxford, 2012
> 
> [12] James M Flegal, Murali Haran, and Galin L Jones. Markov chain monte carlo: Can we trust the third significant figure? Statistical Science, pages250–260, 2008.
> 
> [13] Andrew Gelman, John B Carlin, Hal S Stern, and Donald B Rubin. Bayesian data analysis. texts in statistical science series, 2004

---

<div class="post-metadata">

**Author:** ![rikh](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rikh/32/204104_2.png) [@rikh](https://discourse.julialang.org/u/rikh)\
**Post date:** [February 22, 2021, 10:06am UTC](https://discourse.julialang.org/t/what-is-the-interpretation-of-turings-std-naive-se-mcse/52252/3 "2021-02-22T10:06:22Z")

</div>

The naive Standard Error (naive\_se), according to Rufo on [StackExchange](https://stats.stackexchange.com/a/403617/287164), is

> a measure of the computational MCMC error for the estimation of the posterior expected value of a parameter.
> 
> […]
> 
> If we dig a little bit in the R functions `summary.mcmc.list` from the package `coda` (which uses function `safespec0` and this one uses `spectrum0.ar` in its turn) we find that the definition of the naive SE is:
> 
> SE\_{Naive} = \sqrt{\frac{Var(X)}{C \cdot S}}
> 
> with C being the number of run chains, X = \{X^{(c)}\} being the vector of posterior samples from a certain parameter (concatenation all the chains, c \in 1, ..., C), and S being the length (the number of iterations) of each chain.

r\_hat seems to be the Gelman-Rubin \hat{R}, but I’m not sure what is exactly implemented by Turing since the equation has changed a [few times](https://stats.stackexchange.com/q/348984/287164).

Finally, Cameron Pfiffer [said](https://youtu.be/Jr6HcyHK_Q4?t=853) you shouldn’t rely on `ess`, but instead on `r_hat`. `r_hat` is more reliable.

McElreath ([2020](https://doi.org/10.1201/9780429029608)) summarizes `ess` and `rhat` as `ess` being “a crude estimate of the number of independent samples you managed to get. Rhat (\hat{R}) is an indicator of the convergence of the Markov chains to the target distribution. It should approach 1.00 from above, when all is well.”