High frame rate and resolution from a Raspberry Pi camera

Specific Domains Maker
, ,
1

Has anyone been able to fetch frames from a Raspberry Pi camera (i.e. an RPI with a NoIR Pi Cam connected to it with a ribbon cable) at a relatively high frame rate and resolution?

I realize that “relatively high frame rate and resolution” is literally a relative term, but has anyone been able to change the frames’ temporal and spatial resolutions while also using VideoIO.jl?

While it would be golden to achieve this with Julia alone, I’d be happy to hear any success stories you might have.

1 Like
2

To add some context and details, I’ve managed to get about 7 FPS on

julia> versioninfo()
Julia Version 1.7.2
Commit bf53498635 (2022-02-06 15:21 UTC)
Platform Info:
  OS: Linux (arm-linux-gnueabihf)
  CPU: ARMv7 Processor rev 3 (v7l)
  WORD_SIZE: 32
  LIBM: libopenlibm
  LLVM: libLLVM-12.0.1 (ORCJIT, cortex-a72)

with VideoIO.jl v0.9.5 (due to issue https://github.com/JuliaIO/VideoIO.jl/issues/346).

Here is some code I use to benchmark this:

using VideoIO, UnicodePlots

function benchmark(cam)
    img = read(cam)
    t = zeros(100)
    for i in eachindex(t)
        read!(cam, img)
        t[i] = Base.time()
    end
    fps = 1 ./ diff(t)
    filter!(<(50), fps) # filter out all the unrealisticly high FPSs
    println(histogram(fps))
end

for transcode in (true, false), target_format in (nothing, VideoIO.AV_PIX_FMT_GRAY8)
    cam = opencamera(; transcode, target_format)
    println("Transcode: $transcode Format: $target_format")
    benchmark(cam)
    close(cam)
end
3

I’ve managed phenomenal performance by piping YUV frames from libcamera:

3280×2464 @ 15 fps
1640×1232 @ 40 fps
1640×922 @ 45 fps
1920×1080 @ 45 fps
1280×720 @ 45 fps
480×640 @ 100-300 fps

My implementation is pretty basic (let me know if you see any possible improvements), but seems to work:

using ImageCore

function create_buffer(w, h)
    w2 = 64ceil(Int, w/64) # dimension adjustments to hardware restrictions
    h2 = 32ceil(Int, h/32)
    nb = Int(w2*h2*3//2) # total number of bytes per frame
    return (w2, h2, Vector{UInt8}(undef, nb))
end

struct Camera
    o::Base.Process
    buff::Vector{UInt8}
    img # a reshaped view into the bytes buffer 
    function Camera(w, h, fps)
        w2, h2, buff = create_buffer(w, h)
        b = view(buff, 1:w2*h2)
        Y = reshape(b, w2, h2)
        img = colorview(Gray, normedview(view(Y, 1:w, 1:h)))
        cmd = `libcamera-vid -n --framerate $fps --width $w --height $h -t 0 --codec yuv420 -o -` # I imagine that there might be a number of arguments improving things here, or tailoring it to what the user needs/wants
        o = open(cmd) # to "close" the camera just `kill(c.o)`
        new(o, buff, img)
    end
end

function read_frame(c::Camera)
    read!(c.o, c.buff) # not sure if `readbytes!` might be better here...
    return c.img
end

I got a lot of help from @therealdavidp, a friendly Raspberry Pi Engineer & Forum Moderator on the Raspberry Pi forum (see the actual post here).

To me at least, this highlights the fact that both VideoIO.jl and libcamera can fetch frames from a camera, leaving the question of which is better suited for the job unclear.

I’m excited to see how testing this on the new Raspbian (ARMv8 64-bit) will fair…!

2 Likes