Slow appendencode on Raspberry Pi

General Usage
1

On a Raspberry Pi in the following MWE:

using VideoIO, Dates, BenchmarkTools
cam = VideoIO.opencamera()
img = read(cam)
filename = "video.mp4"
framerate = 200
props = [:priv_data => ("crf"=>"0","preset"=>"ultrafast")]
encoder = prepareencoder(img, framerate=framerate, AVCodecContextProperties=props)
io = Base.open("temp.stream","w")
@btime appendencode!($encoder, $io, $img, 1)

it takes ~430 ms to encode a single frame to a video stream. On my PC it takes ~7 ms – 60 times faster video encoding.

This is a shame because it means we can’t use RPIs to capture, process, and encode live video in real time with a frame rate higher than ~2.

Maybe some of the RPI wizards can figure out what’s going on here?

Thanks in advance!

The RPI involved:

julia> versioninfo()
Julia Version 1.5.2
Commit 539f3ce* (2020-09-23 23:17 UTC)
Platform Info:
  OS: Linux (arm-linux-gnueabihf)
  CPU: ARMv7 Processor rev 3 (v7l)
  WORD_SIZE: 32
  LIBM: libopenlibm
  LLVM: libLLVM-9.0.1 (ORCJIT, cortex-a72)
2

Coincidentally, I was just trying something similar yesterday. I found that VideoIO by default transcodes into RGB (from the more usual YUV format that webcams produce) which slows down the data capture quite a bit. You could try disabling the transcoding (since it’s unnecessary if you are just piping out to a file) with the transcode=false in the opencamera call. However, that hits some only partially implemented code paths but I put a PR to VideoIO with some fixes in https://github.com/JuliaIO/VideoIO.jl/pull/277 but they could definitely use more testing.

That said, I’ve just noticed you aren’t actually timing the data acquisition…

On the encoding side, you could try to use the hardware accelerated encoder on the Pi - it’s the h264_omx codec in FFMPEG (see, e.g., https://askubuntu.com/questions/1224437/how-to-use-hardware-encoding-in-ffmpeg-h264-omx-on-raspberry-pi-4-running-ubun). I don’t know if it’s compiled into the Julia binaries though. It might be possible to coerce VideoIO into using a different binary(?) that you’ve compiled yourself.

1 Like
3

Wow, sounds promising! I think I can copypaste a build-script from BinaryBuilder.jl to build a version of ffmpeg that uses the h264_omx codec instead and then fork VideoIO and change its dependency to use that FFMPEG_jll instead?

Ideally, installing VideoIO would involve checking which system it’s installed on and changing its dependency accordingly… But that sounds like science fiction.

I’ll give it a try. Currently it’s the only viable solution I have to move forward on this front.

Thank you so much!!!

4

You mentioned you needed testing, on my RPI this:

using VideoIO, BenchmarkTools
cam = VideoIO.opencamera()
img = read(cam)
@btime read!($cam, $img)

results in 11.039 ms.
Using your PR, and transcode=false results in 845.115 μs! A ~13 times speedup! So that PR sounds amazing :grinning:

1 Like