# Raspberry Pi record video to file and pipe frames

**URL:** <https://discourse.julialang.org/t/raspberry-pi-record-video-to-file-and-pipe-frames/50493>\
**Category:** General Usage\
**Tags:** question, video\
**Created:** [November 20, 2020, 12:04pm UTC](https://discourse.julialang.org/t/raspberry-pi-record-video-to-file-and-pipe-frames/50493 "2020-11-20T12:04:56Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![yakir12](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yakir12/32/297_2.png) [@yakir12](https://discourse.julialang.org/u/yakir12)\
**Post date:** [November 20, 2020, 12:04pm UTC](https://discourse.julialang.org/t/raspberry-pi-record-video-to-file-and-pipe-frames/50493/1 "2020-11-20T12:04:56Z")

</div>

I’m trying to record a video to a file while at the same time, extract frames from the video stream, on a Raspberry Pi with a Pi Cam.  
If anyone has a better idea on how to do this, please let me know.  
Here’s what I got now:

```julia
w,h = (640, 480) # image dimensions
bytesperframe = w * h # number of bytes in the image
camera_cmd = `raspivid -w $w -h $h --output - --timeout 0 --nopreview` # calling raspivid and piping out the stream
tee_cmd = `tee test_video.h264` # split the stream into 2, one gets saved to file
frame_cmd = `ffmpeg -f h264 -i pipe: -r 1 -f image2pipe -` # and one gets processed by ffmpeg for me to extract frames from
io = open(pipeline(camera_cmd, tee_cmd, frame_cmd)) # start it all
bytes = read(io, bytesperframe) # read one frame
img = reshape(bytes, w, h) # reshape
read!(io, img) # faster 

```

The problem with this is that whenever I `close(io)` I get `Broken pipe` errors, which makes sense, cause how on earth could this piped process stop elegantly? The consequences are that sometimes the video file is corrupt or only partially saved.

My goal is to be able to have a video file recording, while at the same time access (in “real time”) frames from that video so that I can process them. The results of that processing is then served with `JSServe` for diagnostics viewed by users, again, in “real time” (by that I mean within a second or so).

Thanks in advance!

---

<div class="post-metadata">

**Author:** ![yakir12](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yakir12/32/297_2.png) [@yakir12](https://discourse.julialang.org/u/yakir12)\
**Post date:** [November 20, 2020, 4:32pm UTC](https://discourse.julialang.org/t/raspberry-pi-record-video-to-file-and-pipe-frames/50493/2 "2020-11-20T16:32:48Z")

</div>

The solution seems to be using `VideoIO.appendencode`, where I can grab images off of the camera, do what I will with them, and encode them to a video file at the same time. Brilliant! But… The encoding process on the RPI is about 60 times slower than on a PC, resulting in a FPS of about 2. More details here:

> [@Slow appendencode on Raspberry Pi](https://discourse.julialang.org/t/slow-appendencode-on-raspberry-pi/50504):
>
> On a Raspberry Pi in the following MWE: using VideoIO, Dates, BenchmarkTools cam = VideoIO.opencamera() img = read(cam) filename = "video.mp4" framerate = 200 props = [:priv\_data =\> ("crf"=\>"0","preset"=\>"ultrafast")] encoder = prepareencoder(img, framerate=framerate, AVCodecContextProperties=props) io = Base.open("temp.stream","w") @btime appendencode!($encoder, $io, $img, 1) it takes ~430 ms to encode a single frame to a video stream. On my PC it takes ~7 ms – 60 times faster video encoding. …

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [November 21, 2020, 8:32am UTC](https://discourse.julialang.org/t/raspberry-pi-record-video-to-file-and-pipe-frames/50493/3 "2020-11-21T08:32:34Z")

</div>

> [@yakir12](#):
>
> The encoding process on the RPI is about 60 times slower than on a PC, resulting in a FPS of about 2.

I ran into the same problem, and have been wondering if it is possible to just stream raw video data to a server (eg on the same WLAN) that does the encoding, but I have not figured this out yet.

---

<div class="post-metadata">

**Author:** ![yakir12](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yakir12/32/297_2.png) [@yakir12](https://discourse.julialang.org/u/yakir12)\
**Post date:** [March 1, 2021, 8:27am UTC](https://discourse.julialang.org/t/raspberry-pi-record-video-to-file-and-pipe-frames/50493/4 "2021-03-01T08:27:00Z")

</div>

Hi @Tamas_Papp!

I’ve made some progress here, but not quite all the way. First, here is a functioning example of how to save a video file while at the same time _save to file_ the last frame (in these examples I crop the image to a square and resize it, you can ignore those parts):

```julia
using FFMPEG_jll

const SZ = 640 # width and height of the images
const FPS = 5 # frames per second
const IMGPATH = "frame.png" # this is the image file of the last frame

start_camera(file) = ffmpeg() do exe
    run(`$exe -y -hide_banner -loglevel error -f v4l2 -i /dev/video0 -filter_complex '[0:v]crop=in_h:in_h,split=2[out1][out2]' -map '[out1]' $file -map '[out2]' -s $(SZ)x$SZ -r $FPS -update 1 $IMGPATH`, wait = false)
end

p = start_camera("tmp.mp4")
sleep(10) # record for 10 seconds
kill(p) # finish

```

This works very well, but it writes to disk each frame, so there’s a lot of unnecessary IO operations.

Instead, here is an in-memory version, but there’s a but at the end (the following `readpngdata` was shamelessly taken from @Per’s [readpngdata](https://github.com/perrutquist/FFmpegPipe.jl/blob/cc2d73acfa8ce55e3e4e53b8264c94477fa0bce3/src/FFmpegPipe.jl#L74)):

```julia
using FFMPEG_jll, FileIO

function readpngdata(io) 
    blk = 65536;
    a = Array{UInt8}(undef, blk)
    readbytes!(io, a, 8)
    if view(a, 1:8) != magic(format"PNG")
        error("Bad magic.")
    end
    n = 8
    while !eof(io)
        if length(a)<n+12
            resize!(a, length(a)+blk)
        end
        readbytes!(io, view(a, n+1:n+12), 12)
        m = 0
        for i=1:4
            m = m<<8 + a[n+i]
        end
        chunktype = view(a, n+5:n+8)
        n=n+12
        if chunktype == codeunits("IEND")
            break
        end
        if length(a)<n+m
            resize!(a, max(length(a)+blk, n+m+12))
        end
        readbytes!(io, view(a, n+1:n+m), m)
        n = n+m
    end
    resize!(a,n)
    return a
end

const SZ = 640 # width and height of the images
const FPS = 5 # frames per second
const IMG = Ref{Vector{UInt8}}() # a container for the last frame

start_camera(file) = ffmpeg() do exe
    io = open(`$exe -y -hide_banner -loglevel error -f v4l2 -s 1920x1080 -i /dev/video0 -filter_complex '[0:v]crop=in_h:in_h,split=2[out1][out2]' -map '[out1]' $file -map '[out2]' -s $(SZ)x$SZ -r $FPS -vcodec png -f image2pipe -`)
    @async while process_running(io) # repeatedly update the IMG container with the last frame
        IMG[] = readpngdata(io)
    end
    return io
end

p = start_camera("tmp.mp4")

```

While this works, I cannot close, kill, or terminate the process gracefully, and all of my attempts to do so (calling `kill(p)` twice terminates it) result in a corrupt video file.

I’ll keep stabbing at this, but if you can figure out a way to terminate this `Process`, that would be golden. Maybe there is an intrinsic difficulty in terminating a process that is split like that (it’s split by what I think is `ffmpeg`’s version of `tee` is, see more details [here](https://trac.ffmpeg.org/wiki/Creating%20multiple%20outputs)), or maybe it’s solvable. Alternatively, if we could somehow send a `'q'` to the process then it should terminate gracefully (but I’m not sure about that since this process isn’t from a `run`, it’s from a `open`).

Any feedback about this would be greatly appreciated!