# Fisher's test p-value results appear to differ from matlab, R

**URL:** <https://discourse.julialang.org/t/fishers-test-p-value-results-appear-to-differ-from-matlab-r/10630>\
**Category:** Statistics\
**Created:** [May 1, 2018, 12:28am UTC](https://discourse.julialang.org/t/fishers-test-p-value-results-appear-to-differ-from-matlab-r/10630 "2018-05-01T00:28:35Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![ScottWatkins](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/scottwatkins/32/7862_2.png) [@ScottWatkins](https://discourse.julialang.org/u/ScottWatkins)\
**Post date:** [May 1, 2018, 12:28am UTC](https://discourse.julialang.org/t/fishers-test-p-value-results-appear-to-differ-from-matlab-r/10630/1 "2018-05-01T00:28:35Z")

</div>

Hi,  
I was using HypothesisTests.jl and noticed minor, but possibly important differences in the p-value estimate between julia and two other packages when using Fisher’s exact test. In this example, matlab and R users might claim a significant result, whereas the julia user might not. This is a fairly common test, and it would be great if there was agreement. Thanks for considering this comment for discussion.  
Scott

julia (v0.6):

```julia
julia> FisherExactTest(59, 335, 172, 1366)
Fisher's exact test
-------------------
Population details:
    parameter of interest: Odds ratio
    value under h_0: 1.0
    point estimate: 1.3984544219625261
    95% confidence interval: (0.9980930945998796, 1.9393947540537153)

Test summary:
    outcome with 95% confidence: fail to reject h_0
    **two-sided p-value: 0.051329212328076565 <------------------------**

Details:
    contingency table:
         59 335
        172 1366

```

matlab:

```julia
>> x = table([59;172],[335;1366])
x =
  2×2 table
    Var1 Var2
    ________
     59 335
    172 1366

>> [h,p,stats]=fishertest(x,'Tail','both','Alpha',0.95)

h =
  logical
   1

**p =**
 **0.045036387203992 <--------------------------------**

stats = 
  struct with fields:
             OddsRatio: 1.398715723707046
    ConfidenceInterval: [1.384515635855077 1.413061452741955]

```

R:

```julia
> x = matrix(c(59,172,335,1366), nrow = 2)
> x
     [,1] [,2]
[1,] 59 335
[2,] 172 1366
> fisher.test(x,alternative = "two.sided")

	Fisher's Exact Test for Count Data

data: x
**p-value = 0.04503639 <------------------------------**
alternative hypothesis: true odds ratio is not equal to 1
95 percent confidence interval:
 0.9980904309 1.9393926010
sample estimates:
 odds ratio 
1.398453903

```

---

<div class="post-metadata">

**Author:** ![rdeits](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rdeits/32/286_2.png) [@rdeits](https://discourse.julialang.org/u/rdeits)\
**Post date:** [May 1, 2018, 1:13am UTC](https://discourse.julialang.org/t/fishers-test-p-value-results-appear-to-differ-from-matlab-r/10630/2 "2018-05-01T01:13:07Z")

</div>

I’m afraid I can’t help with the statistics question, but it will be easier for others to read your post and help you if you [quote your code with backticks](https://discourse.julialang.org/t/psa-how-to-quote-code-with-backticks/7530).

---

<div class="post-metadata">

**Author:** ![Nosferican](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nosferican/32/9275_2.png) [@Nosferican](https://discourse.julialang.org/u/Nosferican)\
**Post date:** [May 1, 2018, 1:32am UTC](https://discourse.julialang.org/t/fishers-test-p-value-results-appear-to-differ-from-matlab-r/10630/3 "2018-05-01T01:32:29Z")

</div>

Might look into it, but from a quick look, there are adjustments to the test for GLM such as those for count data which lead to the proper test being one-tailed. It might be that software usually take into account those cases as opposed to Julia which might not be aware that it is a count model and the user requests a two-tail version.

---

<div class="post-metadata">

**Author:** ![andreasnoack](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/andreasnoack/32/27_2.png) [@andreasnoack](https://discourse.julialang.org/u/andreasnoack)\
**Post date:** [May 1, 2018, 6:39am UTC](https://discourse.julialang.org/t/fishers-test-p-value-results-appear-to-differ-from-matlab-r/10630/4 "2018-05-01T06:39:53Z")

</div>

For asymmetric distributions, p-values for two-sided alternatives are not well defined. It has been discussed couple of times before. See [https://github.com/JuliaStats/HypothesisTests.jl/issues/98#issuecomment-301532196](https://github.com/JuliaStats/HypothesisTests.jl/issues/98#issuecomment-301532196) and the reference there. See also [the doc string](http://juliastats.github.io/HypothesisTests.jl/stable/methods.html#HypothesisTests.pvalue-Tuple%7BHypothesisTests.FisherExactTest%7D) for the `pvalue` method which states that

```julia
For tail = :both, possible values for method are:

    • :central (default): Central interval, i.e. the p-value is two times the minimum of the
        one-sided p-values.

    • :minlike: Minimum likelihood interval, i.e. the p-value is computed by summing all
        tables with the same marginals that are equally or less probable:

          p_ω = \sum_{f_ω(i)≤ f_ω(a)} f_ω(i)

```

so you can get the p-value used by R by specifying `method`, i.e.

```julia
julia> pvalue(x, method = :minlike)
0.04503638720401171

```

This value is not more correct than the other one.

---

<div class="post-metadata">

**Author:** ![ScottPJones](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/scottpjones/32/146_2.png) [@ScottPJones](https://discourse.julialang.org/u/ScottPJones)\
**Post date:** [May 1, 2018, 3:54pm UTC](https://discourse.julialang.org/t/fishers-test-p-value-results-appear-to-differ-from-matlab-r/10630/5 "2018-05-01T15:54:14Z")

</div>

> [@andreasnoack](#):
>
> This value is not more correct than the other one.

I’m curious, does R or Matlab give the option of calculating the `:central` value that is the default for Julia?

---

<div class="post-metadata">

**Author:** ![ScottWatkins](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/scottwatkins/32/7862_2.png) [@ScottWatkins](https://discourse.julialang.org/u/ScottWatkins)\
**Post date:** [May 1, 2018, 4:04pm UTC](https://discourse.julialang.org/t/fishers-test-p-value-results-appear-to-differ-from-matlab-r/10630/6 "2018-05-01T16:04:17Z")

</div>

Robin, thanks for the advice on posting. Jose, thanks for advice on modeling. And Andreas, thank you for a great explanation. I was easily able to reproduce the matlab/R results. For a newbie like me who just used the inline REPL help, an additional sentence in the REPL help giving a little more detail on how alternative p-values can be obtained would be helpful!

---

<div class="post-metadata">

**Author:** ![aalexandersson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/aalexandersson/32/2927_2.png) [@aalexandersson](https://discourse.julialang.org/u/aalexandersson)\
**Post date:** [May 1, 2018, 4:25pm UTC](https://discourse.julialang.org/t/fishers-test-p-value-results-appear-to-differ-from-matlab-r/10630/7 "2018-05-01T16:25:48Z")

</div>

> [@ScottPJones](#):
>
> I’m curious, does R or Matlab give the option of calculating the :central value that is the default for Julia?

R does (my first post, I hope I got it right!):

```julia
> library(exact2x2)
> x = matrix(c(59,172,335,1366), nrow = 2)
> x
     [,1] [,2]
[1,] 59 335
[2,] 172 1366
> 
> exact2x2(x,tsmethod="central")

	Central Fisher's Exact Test

data: x
p-value = 0.05133
alternative hypothesis: true odds ratio is not equal to 1
95 percent confidence interval:
 0.9980904 1.9393926
sample estimates:
odds ratio 
  1.398454

```