# Step Detection

**URL:** <https://discourse.julialang.org/t/step-detection/96217>\
**Category:** General Usage\
**Tags:** signal-processing\
**Created:** [March 17, 2023, 12:05am UTC](https://discourse.julialang.org/t/step-detection/96217 "2023-03-17T00:05:30Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![ellocco](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ellocco/32/31331_2.png) [@ellocco](https://discourse.julialang.org/u/ellocco)\
**Post date:** [March 17, 2023, 12:05am UTC](https://discourse.julialang.org/t/step-detection/96217/1 "2023-03-17T00:05:30Z")

</div>

I would like to [detect steps](https://en.wikipedia.org/wiki/Step_detection) in a signal.  
I wonder, if a Julia package exists that can do this job. I found the Python package [`ruptures`](https://centre-borelli.github.io/ruptures-docs/) that does the job, but it has two shortcomings:

- It is slow
- It can not handle larger vectors

It took a while before I understood why I face all the time crahses, the reason was simply that my data  
vector is to long (in my case it contains more then 3e5 elements).  
Is there another package that can do the same also on longer vectors?

Here is a code snippet for those how are interested to experiment with this package:

```julia
using PyCall, NumPyArrays
rpt = PyCall.pyimport("ruptures")
# --- signal:
n_ = 1000
ones_ = ones(n_,1)
zeros_ = zeros(n_,1)
x_ = vcat(ones_, zeros_, ones_, zeros_, ones_, zeros_, ones_, zeros_, ones_, zeros_, ones_, zeros_, 
    ones_, zeros_, ones_, zeros_, ones_, zeros_, ones_, zeros_, ones_, zeros_, ones_, zeros_, 
    )
signal_ = NumPyArray(x_)

# --- detection
algo = rpt.Pelt(model="rbf").fit(signal_)
result = algo.predict(pen=2)

```

---

<div class="post-metadata">

**Author:** ![SteffenPL](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/steffenpl/32/206270_2.png) [@SteffenPL](https://discourse.julialang.org/u/SteffenPL)\
**Post date:** [March 17, 2023, 2:19am UTC](https://discourse.julialang.org/t/step-detection/96217/2 "2023-03-17T02:19:33Z")

</div>

I’m not an expert in these things, just some search results which might help:

- This seems to be a similar question: [Looking for a time-series jump test package](https://discourse.julialang.org/t/looking-for-a-time-series-jump-test-package/56773)
- Maybe also this [GitHub - CNelias/ChangePointDetection.jl: Julia implementation of the LSDD change point detection method.](https://github.com/CNelias/ChangePointDetection.jl)

---

<div class="post-metadata">

**Author:** ![ellocco](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ellocco/32/31331_2.png) [@ellocco](https://discourse.julialang.org/u/ellocco)\
**Post date:** [March 17, 2023, 2:29am UTC](https://discourse.julialang.org/t/step-detection/96217/3 "2023-03-17T02:29:53Z")

</div>

> [@SteffenPL](#):
>
> is [GitHub - CNelias/ChangePointDetection.jl: Julia implementation of the LSDD](https://github.com/CNelias/ChangePointDetection.jl)

Yes, I saw this package, but it seems to be not any longer maintained, maybe I will give it a try anyhow.  
Currently, I code my own function, the logic is the same, you compare two neighboring moving windows / cohorts, and when both differ significantly you have probably detected a step.

---

<div class="post-metadata">

**Author:** ![SteffenPL](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/steffenpl/32/206270_2.png) [@SteffenPL](https://discourse.julialang.org/u/SteffenPL)\
**Post date:** [March 17, 2023, 2:31am UTC](https://discourse.julialang.org/t/step-detection/96217/4 "2023-03-17T02:31:41Z")

</div>

For detecting the peaks, you could use [Peaks.jl](https://github.com/halleysfifthinc/Peaks.jl)

---

<div class="post-metadata">

**Author:** ![baggepinnen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/baggepinnen/32/693_2.png) [@baggepinnen](https://discourse.julialang.org/u/baggepinnen)\
**Post date:** [March 17, 2023, 4:23am UTC](https://discourse.julialang.org/t/step-detection/96217/5 "2023-03-17T04:23:25Z")

</div>

There are a few utilities for change-point detection using matrix profiles implemented in [GitHub - baggepinnen/MatrixProfile.jl: Time-series analysis using the Matrix profile in Julia](https://github.com/baggepinnen/MatrixProfile.jl#segmentation--change-point-detection)

The matrix profile is quite useful for a lot of different time series tasks, but can be expensive to compute for very long vectors.

---

<div class="post-metadata">

**Author:** ![rafael.guerra](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rafael.guerra/32/216610_2.png) [@rafael.guerra](https://discourse.julialang.org/u/rafael.guerra)\
**Post date:** [March 17, 2023, 6:06am UTC](https://discourse.julialang.org/t/step-detection/96217/6 "2023-03-17T06:06:57Z")

</div>

Linking [related thread](https://piecewise-constant-fitting).

---

<div class="post-metadata">

**Author:** ![ellocco](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ellocco/32/31331_2.png) [@ellocco](https://discourse.julialang.org/u/ellocco)\
**Post date:** [March 17, 2023, 7:02am UTC](https://discourse.julialang.org/t/step-detection/96217/7 "2023-03-17T07:02:11Z")

</div>

> [@baggepinnen](#):
>
> baggepinnen/MatrixProfile.jl: Time-series analysis using the Matrix profile in Julia

Thanks! - I saw also your package, but I am struggling to instrument it for my use case:  
a set-point changes from time to time up-and-down and other variables follow in time,  
some with a time shift, others without, what would be the practical approach to do separate cohorts with similar value, cutting out the transition periods with your package?

---

<div class="post-metadata">

**Author:** ![baggepinnen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/baggepinnen/32/693_2.png) [@baggepinnen](https://discourse.julialang.org/u/baggepinnen)\
**Post date:** [March 17, 2023, 7:14am UTC](https://discourse.julialang.org/t/step-detection/96217/8 "2023-03-17T07:14:07Z")

</div>

Do you have a good dynamical model of how the system behaves? If so, something like a Kalman filter is likely a better approach for multivariate time series. By monitoring the prediction error of the Kalman filter, you detect a step when the prediction error becomes large in terms of the posterior covariance matrix. A quick google search returned tons of results for

> kalman filter for change point detection

this review article covers this use case

> **[1908.07136.pdf](https://arxiv.org/pdf/1908.07136.pdf)**
>
> 146.54 KB

Kalman filters, and other state estimators, are implemented [in several julia packages](https://juliahub.com/ui/Search?q=kalmanfilter&type=docs).

---

<div class="post-metadata">

**Author:** ![ellocco](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ellocco/32/31331_2.png) [@ellocco](https://discourse.julialang.org/u/ellocco)\
**Post date:** [March 17, 2023, 7:20am UTC](https://discourse.julialang.org/t/step-detection/96217/9 "2023-03-17T07:20:38Z")

</div>

Thanks, for the link! 🙂  
Yesterday, I also stumbled a few times over the key word calman filter.  
Today I need something quick and dirty … I have an idea about the length of the transition periods and I have an idea about the min duration of steady periods.  
 ![step_change](https://global.discourse-cdn.com/julialang/original/3X/2/7/27f56908f88d518191773b6fb8553ad9a5581d0e.png)

I hope I find the time to study also the calman filter approach later on.

---

<div class="post-metadata">

**Author:** ![ellocco](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ellocco/32/31331_2.png) [@ellocco](https://discourse.julialang.org/u/ellocco)\
**Post date:** [March 17, 2023, 7:55am UTC](https://discourse.julialang.org/t/step-detection/96217/10 "2023-03-17T07:55:55Z")

</div>

> [@SteffenPL](#):
>
> this [GitHub - CNelias/ChangePointDetection.jl: Julia implementation of the LSDD change point detection method.](https://github.com/CNelias/ChangePointDetection.jl)

This one is really slow!!! ☹

---

<div class="post-metadata">

**Author:** ![ellocco](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ellocco/32/31331_2.png) [@ellocco](https://discourse.julialang.org/u/ellocco)\
**Post date:** [March 17, 2023, 9:16am UTC](https://discourse.julialang.org/t/step-detection/96217/11 "2023-03-17T09:16:44Z")

</div>

> [@ellocco](#):
>
> Today I need something quick and dirty …

Here my quick and dirty approach, with the tuning parameter `indx_offset` I can move the step indicator in the right direction, if needed:

```julia
# x_ and y_ are given, x_ might be the time vector.

function step_changes(_signal::AbstractVector; _cohort_length::Int=100, Δthrhld::Number=0.0125, _step_gap::Int=5, _Δstep::Int=100, indx_offset::Int=10)
    _steps = []; wait_conter = 0
    for i_step = 1:length(_signal) - (2 * _cohort_length + _step_gap)
        wait_conter = max(0, wait_conter - 1)
        cohort_end = i_step + _cohort_length - 1
        cohort_a = _signal[i_step : cohort_end]
        cohort_start = cohort_end + _step_gap
        cohort_b = _signal[cohort_start : cohort_start + _cohort_length - 1]
        Δmean = abs(mean(cohort_a) - mean(cohort_b))
        if Δmean > Δthrhld && wait_conter == 0
            push!(_steps, i_step + _cohort_length - indx_offset)
            wait_conter = _Δstep
        end
    end
    return _steps
end

step_changes_starts = step_changes(y_; _cohort_length = 100, Δthrhld = 0.01, _Δstep = 10000, indx_offset=50)
time_stamps_power_up = x_[step_changes_starts]

step_changes_stops = step_changes(reverse(y_); _cohort_length = 100, Δthrhld = 0.01, _Δstep = 10000, indx_offset=60)
time_stamps_power_down = reverse(x_)[step_changes_stops]

```

By calling the function twice I can distinguish between the start and the stop index of one stair tread.  
For my data it works perfect with the correct parameters:

 ![step_change_detection](https://global.discourse-cdn.com/julialang/original/3X/7/c/7ca9789d3682d9f517f74780e73e36d5658cb6cb.png)