# Calculating Breaths Per Minute from Video Data

**URL:** <https://discourse.julialang.org/t/calculating-breaths-per-minute-from-video-data/51582>\
**Category:** Signal and Image Processing\
**Created:** [December 10, 2020, 3:45am UTC](https://discourse.julialang.org/t/calculating-breaths-per-minute-from-video-data/51582 "2020-12-10T03:45:27Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![RodBiren](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rodbiren/32/16979_2.png) [@RodBiren](https://discourse.julialang.org/u/RodBiren)\
**Post date:** [December 10, 2020, 3:45am UTC](https://discourse.julialang.org/t/calculating-breaths-per-minute-from-video-data/51582/1 "2020-12-10T03:45:27Z")

</div>

I have been able to process video data of myself sleeping using a Wyze camera to produce fairly solid data that I would like to use to calculate breaths per minute.The actual calculation is trivial. I calculate the percent change an image is different from a keyframe where 0 is the same image, and 1% is N/(Width \* Height \* 256). I create a threshold for percent difference and trigger a retaking of the keyframe and a slight pause in the data processing.

Sample Signal:

 ![image](https://global.discourse-cdn.com/julialang/original/3X/b/e/be8ce779b4883d7ed9976204d1e1a210f75a439c.png)

Using a discrete fourier transform I get a good result.

 ![image](https://global.discourse-cdn.com/julialang/original/3X/a/9/a96d54191a22a19aec8c5d95440cae80fbf53def.png)

So I can clearly see the FFT is yielding a string signal, but I am unsure how to convert this properly to a breaths per minute calculation. I capture timestamps of the data as I am processing it as a CSV, but when I take windows of the data the position of the spike in the FFT changes, so I think my understanding of what a discrete fourier transform calculates is naive. I know typical sleeping breaths per minute is around 12 ~20 BPM. But the relationship between the actual frequency and the result of the FFT is abstract. I have wanted to attempt this for awhile, but was inspired to give it a try when I say the most recent [julia video for DFT](https://www.youtube.com/watch?v=g8RkArhtCc4)

I think my next steps should be using DSP.jl, but I have no idea how to use it. The documentation describes the functions, but I am unable to find examples. Main point being I want to know what methods or libraries I can use to pull the dominant frequencies from the signal in real time domain. I am an amateur programmer and signal processor, and haven’t had to do differential equations for about 10 years, but I find it all fascinating and stimulating. Would appreciate any help.

---

<div class="post-metadata">

**Author:** ![Jeff\_Emanuel](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jeff_emanuel/32/15440_2.png) [@Jeff\_Emanuel](https://discourse.julialang.org/u/Jeff_Emanuel)\
**Post date:** [December 10, 2020, 3:22pm UTC](https://discourse.julialang.org/t/calculating-breaths-per-minute-from-video-data/51582/2 "2020-12-10T15:22:36Z")

</div>

From your time sample rate you calculate the Nyquist frequency, then the frequency sample interval is that divided by half (because of positive and negative frequencies) the length of the FFT signal.

---

<div class="post-metadata">

**Author:** ![contradict](https://avatars.discourse-cdn.com/v4/letter/c/ac91a4/32.png) [@contradict](https://discourse.julialang.org/u/contradict)\
**Post date:** [December 10, 2020, 5:11pm UTC](https://discourse.julialang.org/t/calculating-breaths-per-minute-from-video-data/51582/3 "2020-12-10T17:11:41Z")

</div>

For the very narrow question of ‘given the position of this peak in the FFT coefficients, what is the corresponding frequency?’, you can use the `fftfreq` function (available in `FFTW`).

```julia
 julia> Ts = 1. /30 # I am assuming your camera takes pictures at 30 FPS
0.03333333333333333

julia> t = 0:Ts:60
0.0:0.03333333333333333:60.0

julia> Fbreath = 20. / 60. # 20 breaths in 60 seconds
0.3333333333333333

julia> signal = sin.(2π.*t*Fbreath) .+ 0.1*rand(Float64, size(t)); # some fake data

julia> fft_sig = fft(signal);

julia> argmax(abs.(fft_sig))
21

julia> fftfreq(length(signal), 1/Ts)[21] # second argument is the sampling rate
0.3331482509716824

```

---

<div class="post-metadata">

**Author:** ![contradict](https://avatars.discourse-cdn.com/v4/letter/c/ac91a4/32.png) [@contradict](https://discourse.julialang.org/u/contradict)\
**Post date:** [December 10, 2020, 5:27pm UTC](https://discourse.julialang.org/t/calculating-breaths-per-minute-from-video-data/51582/4 "2020-12-10T17:27:16Z")

</div>

For the more complicated question, “How do I find the spectral power of the frequencies in this signal”, you might want to look at a [periodogram](https://docs.juliadsp.org/stable/periodograms/) or a short-time-FFT.

There are also techniques to directly estimate the dominant sinusoidal components of a signal [Estimation - parametric estimation functions · DSP.jl](https://docs.juliadsp.org/stable/estimation/) it would be fun to see how that performs on your data!

---

<div class="post-metadata">

**Author:** ![RodBiren](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rodbiren/32/16979_2.png) [@RodBiren](https://discourse.julialang.org/u/RodBiren)\
**Post date:** [December 10, 2020, 5:28pm UTC](https://discourse.julialang.org/t/calculating-breaths-per-minute-from-video-data/51582/5 "2020-12-10T17:28:42Z")

</div>

Ohhhh, so fftfreq uses the inverse of the sample rate. That makes sense. I will try this! greatly appreciate the guidance.

---

<div class="post-metadata">

**Author:** ![contradict](https://avatars.discourse-cdn.com/v4/letter/c/ac91a4/32.png) [@contradict](https://discourse.julialang.org/u/contradict)\
**Post date:** [December 10, 2020, 5:30pm UTC](https://discourse.julialang.org/t/calculating-breaths-per-minute-from-video-data/51582/6 "2020-12-10T17:30:22Z")

</div>

`sampling rate` is usually measured in `Hz` or `1/s`. Sample period is measured in `s`.

---

<div class="post-metadata">

**Author:** ![RodBiren](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rodbiren/32/16979_2.png) [@RodBiren](https://discourse.julialang.org/u/RodBiren)\
**Post date:** [December 10, 2020, 5:45pm UTC](https://discourse.julialang.org/t/calculating-breaths-per-minute-from-video-data/51582/7 "2020-12-10T17:45:29Z")

</div>

Works great now. I had gone down the path, but the fftfreq function confused me for some reason and I could not figure out the relationship between the result and my expected ranges. That cleared it up for me and looks like fairly accurate data.

 ![image](https://global.discourse-cdn.com/julialang/original/3X/c/d/cd1f36a77ba9dac52f1da278451532e2e0165f5f.png)

I have to ignore the first element and the module noise appears as an insanely high signal. Thank you so much for clearing that up!

---

<div class="post-metadata">

**Author:** ![johnh](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/johnh/32/3615_2.png) [@johnh](https://discourse.julialang.org/u/johnh)\
**Post date:** [December 10, 2020, 5:52pm UTC](https://discourse.julialang.org/t/calculating-breaths-per-minute-from-video-data/51582/8 "2020-12-10T17:52:53Z")

</div>

@RodBiren May I Ask what the application is here? Maybe monitoring vulnerable or elderly people at home?  
Then again people normally sleep under covers! It would be very interesting to hear what the application is, if you can say of course - no commercial secrets should be given.

---

<div class="post-metadata">

**Author:** ![Jeff\_Emanuel](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jeff_emanuel/32/15440_2.png) [@Jeff\_Emanuel](https://discourse.julialang.org/u/Jeff_Emanuel)\
**Post date:** [December 10, 2020, 6:05pm UTC](https://discourse.julialang.org/t/calculating-breaths-per-minute-from-video-data/51582/9 "2020-12-10T18:05:38Z")

</div>

> [@RodBiren](#):
>
> the first element and the module noise appears as an insanely high signal

DC bias? Subtract the mean of your input signal to avoid that.

---

<div class="post-metadata">

**Author:** ![RodBiren](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rodbiren/32/16979_2.png) [@RodBiren](https://discourse.julialang.org/u/RodBiren)\
**Post date:** [December 10, 2020, 6:20pm UTC](https://discourse.julialang.org/t/calculating-breaths-per-minute-from-video-data/51582/10 "2020-12-10T18:20:32Z")

</div>

No real application for commercial use or anything. Breaths per minute can be used to estimate sleep stage/quality. The Wyze camera is cheap as dirt and supports Infrared, so really it is just a cheap, albeit creepy, method of tracking my sleep.

Using covers does not actually present much of an issue. My background is with OpenCV C++ and you would be surprised how accurate the cameras really are at seeing small changes.

Total signal for a few hours sleep looks like this

 ![image](https://global.discourse-cdn.com/julialang/original/3X/c/2/c239ac42e1e218dfd9da42cfa258668c59006194.png)

Being an amateur signal processor I just cap the changes at 1% so I end up with a bunch of floating Sin waves broken up by periods where I clearly readjusted and moved. Each locations has varied amplitude as a result of my position and how much rise and fall the covers allow to be viewed. But at no point was the signal totally gone, so it made for a reasonably accurate measurement device.