# Bayes Testing

Here is a version of the JZS t-test, I need to wirte out the fromula of Student’s t-test, or find a pacakge, that only expresses the t-value, and not a full description.

``````mu=mean(group1) # the group that is not manipulated
t=(mean(x)*null_hypothesis)/(std(x)/sqrt(N))
N=length(x)
B=(1+(t^2/v)^inv(N/2))/
inv(sqrt(2*pi*g))*g^(-3/2)*exp(-1/(2*g)))),0,inf)
``````

but I don’t know what g means

Bayesian t-test is more easy, you can try to convert a Stan implementation of John Kruschke’s Bayesian Estimation Supersedes the t-test (BEST), in John K. Kruschke, “Bayesian Estimation Supersedes the t test,” Journal of Experimental Psychology 142, no. 2 (May 2013): 573-603, doi:10.1037/a0029146.

This should be easy to do in Turing

``````// Adapted from code by Michael Clark
// https://github.com/m-clark/Miscellaneous-R-Code/blob/master/ModelFitting/Bayesian/rstant_testBEST.R

// Stuff coming in from R
data {
int<lower=1> N;  // Sample size
int<lower=2> n_groups;  // Number of groups
vector[N] y;  // Outcome variable
int<lower=1, upper=n_groups> group_id[N];  // Group variable
}

// Stuff to transform in Stan
transformed data {
real mean_y;
mean_y = mean(y);
}

// Stuff to estimate
parameters {
vector[2] mu;  // Estimated group means
vector<lower=0>[2] sigma;  // Estimated group sd
real<lower=0, upper=100> nu;  // df for t distribution
}

// Models and distributions
model {
// Priors
// curve(expr = dnorm(mean_y, 2), from = -5, to = 5)
mu ~ normal(mean_y, 2);

// curve(expr = dcauchy(x, location = 0, scale = 1), from = 0, to = 40)
sigma ~ cauchy(0, 1);

// Kruschke uses a nu of exponential(1/29)
// curve(expr = dexp(x, 1/29), from = 0, to = 200)
nu ~ exponential(1.0/29);

// Likelihood
for (n in 1:N){
y[n] ~ student_t(nu, mu[group_id[n]], sigma[group_id[n]]);
}
}

// Stuff to calculate with Stan
generated quantities {
// Mean difference
real mu_diff;

// Effect size; see footnote 1 in Kruschke:2013
// Standardized difference between two means
// See https://en.wikipedia.org/wiki/Effect_size#Cohen's_d
real cohen_d;

// Common language effect size
// The probability that a score sampled at random from one distribution will
// be greater than a score sampled from some other distribution
// See https://janhove.github.io/reporting/2016/11/16/common-language-effect-sizes
real cles;

mu_diff = mu[1] - mu[2];
cohen_d = mu_diff / sqrt(sum(sigma)/2);
cles = normal_cdf(mu_diff / sqrt(sum(sigma)), 0, 1);
}
``````

One more thing? Why not fit two models that represents the competing hypotheses and do a LOO-CV using Pareto Smoothing Importance Sampling (PSIS-LOO)? The LOO calculates the Leave-one-out Cross-Validation of the elpd (expected log pointwise predictive density):

and see which model performs better? You wouldn’t even be restricted to only comparing two competing hypotheses at once. You could compare 10, 20, … all together.

It’s hard to convert frequentist approximations shortcuts (as E.T. Jaynes says “adhockeries”) to Bayesian Equivalent. As Allen Downey says (Bayesian and frequentist results are not the same, ever – Probably Overthinking It):

The posterior distribution represents everything you know about the parameters; if you reduce it to a single number, an interval, or a probability, you lose useful information. In fact, you lose exactly the information that makes the posterior distribution useful in the first place.

It’s like comparing a car and an airplane by driving the airplane on the road. You would conclude that the airplane is complicated, expensive, and not particularly good as a car. But that would be a silly conclusion because it’s a silly comparison. The whole point of an airplane is that it can fly.

2 Likes

Is this in r ? I did see a BEST algorithm written in Python

Stan: https://mc-stan.org/

Here is BEST in Python:
https://github.com/strawlab/best/blob/master/best/init.py

I think I can translate most of this pretty easily, but I need to go over Gaussian KDE, I did find KernelDensity.jl

Also, is NonCentralT, some kind of t-test?

OK, there’s a stan package for Julia, but it still involves downloading STAN. I think the python one is easier to change. I’ll get as far as I can, and message it to you.

Is “cles” the final Bayesian Power (term used in Krushke, 2017)?

Is this a type of Gaussian Mixed Model around a Non-central T distribution?