Note that the references you mention do not give the formula to compute the p-value, but assume you have chosen its level and want to check whether the critical value is consistent with it or not. So it’s not unreasonable to expect the formula to be a bit different. That said it’s definitely worth investigating, possibly throwing an error if we think the function is unreliable at the moment.
An easy way to test this without reading any paper would be to test the distribution of the p-values of null hypothesis generated data (should be uniform [0,1])
This would be a good backup method in any case.
2 Likes
An amusing caveat: How does one test if the p-values distribute Uniform(0,1)
? Using the Kolmogorov-Smirnov test. This is awkwardly circular, but this can be remedied by choosing the number of samples N
for the uniformity test high enough to make it pass reasonable thresholds.
2 Likes