Mann Kendall Test - Sen's Slope

Hello to everyone,

Let’s suppose we have 3-hourly fluxes of solar radiation at surface, for a period 1980-2020.
I want to apply Mann Kendall Test to this time-series and find the Sen’s slope, intercept, p_value and tau.
I am new user and i don’t find any package to calculate this. Have you any Julian suggestions ? ?
I know about python’s pymannkendall, that gives that i want, but when i PyCall it, it gives me error (too large iterator) for the whole time series period. However when i use it for the half period for example, it works fine…

Thank you for your time.

Hello,

just got your email :slight_smile:

As far as I know, there is nothing coded right now in Julia. There is this Matlab Toolbox/code:

The license allows for redistribution in code or binary form provided you keep the copyrights, etc…

It is about 250 lines of code, so might be “straightforward” to recode it from this source?

Hi,
thanks for your advice.
I will start to recode it.

Hello,
Here is the non parametric Mann-Kendall Test and Sen’s slope for Julia:

As you can see input/output is sth like this:
julia> mk(x,y)
(reject_null_hypothesis = false, p_value = 0.4369827573849885, Tau = -0.02374739039665971, slope = -7.613151196044908e-5, intercept = 0.5150870782436969)

BUT, the issue now is that when my dataset is too large this function is too slow or give OutOfMemory Error. Too large i mean the length is ~10,000 in 1GB RAM, or 200,000 for 32 GB.

However, with the same length, the GLM.jl [ lm = fit(LinearModel, @formula(y ~ x),df) ] works fine !
So, i think that with some way, someone, can modify this function in order to win these memory issues.
I would greatly appreciate if anyone could find a solution to this.

Cheers, Michael

2 Likes

Hi Michael,
I was intrigued by your question and tried to get information on how it was implemented. I stumbled upon the following articles:

They seem to indicate additional terms in the definition of the variance about the tied groups of values. (or is it what you mean by your comment in the code indicating the coefficient is not adjusted for ties?)
If you need to modify the variance in that way, that will likely introduce some extra complexity.

I hope it helps.

Eric

Hello,
My definition don’t use tied groups, as i wrote in the comment for coefficient. Truly they modify the variance for tied groups. My “issue” isn’t the ties, but the length of data that when it is more than 10000 i get OutOfMemory Error, while with the same length GLM.jl works fine. So i am looking how to solve this memory issue.
Thank you for your interest and congratulations for HoeffdingD.jl !

Cheers,
Michael