I would like to detect steps in a signal.

I wonder, if a Julia package exists that can do this job. I found the Python package `ruptures`

that does the job, but it has two shortcomings:

- It is slow
- It can not handle larger vectors

It took a while before I understood why I face all the time crahses, the reason was simply that my data

vector is to long (in my case it contains more then 3e5 elements).

Is there another package that can do the same also on longer vectors?

Here is a code snippet for those how are interested to experiment with this package:

```
using PyCall, NumPyArrays
rpt = PyCall.pyimport("ruptures")
# --- signal:
n_ = 1000
ones_ = ones(n_,1)
zeros_ = zeros(n_,1)
x_ = vcat(ones_, zeros_, ones_, zeros_, ones_, zeros_, ones_, zeros_, ones_, zeros_, ones_, zeros_,
ones_, zeros_, ones_, zeros_, ones_, zeros_, ones_, zeros_, ones_, zeros_, ones_, zeros_,
)
signal_ = NumPyArray(x_)
# --- detection
algo = rpt.Pelt(model="rbf").fit(signal_)
result = algo.predict(pen=2)
```

Iâ€™m not an expert in these things, just some search results which might help:

Yes, I saw this package, but it seems to be not any longer maintained, maybe I will give it a try anyhow.

Currently, I code my own function, the logic is the same, you compare two neighboring moving windows / cohorts, and when both differ significantly you have probably detected a step.

1 Like

For detecting the peaks, you could use Peaks.jl

There are a few utilities for change-point detection using matrix profiles implemented in GitHub - baggepinnen/MatrixProfile.jl: Time-series analysis using the Matrix profile in Julia

The matrix profile is quite useful for a lot of different time series tasks, but can be expensive to compute for very long vectors.

Thanks! - I saw also your package, but I am struggling to instrument it for my use case:

a set-point changes from time to time up-and-down and other variables follow in time,

some with a time shift, others without, what would be the practical approach to do separate cohorts with similar value, cutting out the transition periods with your package?

Do you have a good dynamical model of how the system behaves? If so, something like a Kalman filter is likely a better approach for multivariate time series. By monitoring the prediction error of the Kalman filter, you detect a step when the prediction error becomes large in terms of the posterior covariance matrix. A quick google search returned tons of results for

kalman filter for change point detection

this review article covers this use case

Kalman filters, and other state estimators, are implemented in several julia packages.

2 Likes

Thanks, for the link!

Yesterday, I also stumbled a few times over the key word calman filter.

Today I need something quick and dirty â€¦ I have an idea about the length of the transition periods and I have an idea about the min duration of steady periods.

I hope I find the time to study also the calman filter approach later on.

This one is really slow!!!

Here my quick and dirty approach, with the tuning parameter `indx_offset`

I can move the step indicator in the right direction, if needed:

```
# x_ and y_ are given, x_ might be the time vector.
function step_changes(_signal::AbstractVector; _cohort_length::Int=100, Î”thrhld::Number=0.0125, _step_gap::Int=5, _Î”step::Int=100, indx_offset::Int=10)
_steps = []; wait_conter = 0
for i_step = 1:length(_signal) - (2 * _cohort_length + _step_gap)
wait_conter = max(0, wait_conter - 1)
cohort_end = i_step + _cohort_length - 1
cohort_a = _signal[i_step : cohort_end]
cohort_start = cohort_end + _step_gap
cohort_b = _signal[cohort_start : cohort_start + _cohort_length - 1]
Î”mean = abs(mean(cohort_a) - mean(cohort_b))
if Î”mean > Î”thrhld && wait_conter == 0
push!(_steps, i_step + _cohort_length - indx_offset)
wait_conter = _Î”step
end
end
return _steps
end
step_changes_starts = step_changes(y_; _cohort_length = 100, Î”thrhld = 0.01, _Î”step = 10000, indx_offset=50)
time_stamps_power_up = x_[step_changes_starts]
step_changes_stops = step_changes(reverse(y_); _cohort_length = 100, Î”thrhld = 0.01, _Î”step = 10000, indx_offset=60)
time_stamps_power_down = reverse(x_)[step_changes_stops]
```

By calling the function twice I can distinguish between the start and the stop index of one stair tread.

For my data it works perfect with the correct parameters:

1 Like