# High frame rate and resolution from a Raspberry Pi camera

**URL:** <https://discourse.julialang.org/t/high-frame-rate-and-resolution-from-a-raspberry-pi-camera/76815>\
**Category:** Maker\
**Tags:** question, video, raspberry-pi\
**Created:** [February 20, 2022, 7:39pm UTC](https://discourse.julialang.org/t/high-frame-rate-and-resolution-from-a-raspberry-pi-camera/76815 "2022-02-20T19:39:21Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![yakir12](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yakir12/32/297_2.png) [@yakir12](https://discourse.julialang.org/u/yakir12)\
**Post date:** [February 20, 2022, 7:39pm UTC](https://discourse.julialang.org/t/high-frame-rate-and-resolution-from-a-raspberry-pi-camera/76815/1 "2022-02-20T19:39:21Z")

</div>

Has anyone been able to fetch frames from a Raspberry Pi camera (i.e. an RPI with a NoIR Pi Cam connected to it with a ribbon cable) at a _relatively_ high frame rate and resolution?

I realize that “_relatively_ high frame rate and resolution” is literally a _relative_ term, but has anyone been able to **change** the frames’ temporal and spatial resolutions while also using `VideoIO.jl`?

While it would be golden to achieve this with Julia alone, I’d be happy to hear any success stories you might have.

---

<div class="post-metadata">

**Author:** ![yakir12](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yakir12/32/297_2.png) [@yakir12](https://discourse.julialang.org/u/yakir12)\
**Post date:** [February 21, 2022, 9:43am UTC](https://discourse.julialang.org/t/high-frame-rate-and-resolution-from-a-raspberry-pi-camera/76815/2 "2022-02-21T09:43:01Z")

</div>

To add some context and details, I’ve managed to get about 7 FPS on

```julia
julia> versioninfo()
Julia Version 1.7.2
Commit bf53498635 (2022-02-06 15:21 UTC)
Platform Info:
  OS: Linux (arm-linux-gnueabihf)
  CPU: ARMv7 Processor rev 3 (v7l)
  WORD_SIZE: 32
  LIBM: libopenlibm
  LLVM: libLLVM-12.0.1 (ORCJIT, cortex-a72)

```

with VideoIO.jl v0.9.5 (due to issue [https://github.com/JuliaIO/VideoIO.jl/issues/346](https://github.com/JuliaIO/VideoIO.jl/issues/346)).

Here is some code I use to benchmark this:

```julia
using VideoIO, UnicodePlots

function benchmark(cam)
    img = read(cam)
    t = zeros(100)
    for i in eachindex(t)
        read!(cam, img)
        t[i] = Base.time()
    end
    fps = 1 ./ diff(t)
    filter!(<(50), fps) # filter out all the unrealisticly high FPSs
    println(histogram(fps))
end

for transcode in (true, false), target_format in (nothing, VideoIO.AV_PIX_FMT_GRAY8)
    cam = opencamera(; transcode, target_format)
    println("Transcode: $transcode Format: $target_format")
    benchmark(cam)
    close(cam)
end

```

---

<div class="post-metadata">

**Author:** ![yakir12](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yakir12/32/297_2.png) [@yakir12](https://discourse.julialang.org/u/yakir12)\
**Post date:** [February 27, 2022, 6:54pm UTC](https://discourse.julialang.org/t/high-frame-rate-and-resolution-from-a-raspberry-pi-camera/76815/3 "2022-02-27T18:54:30Z")

</div>

I’ve managed phenomenal performance by piping YUV frames from [libcamera](https://www.raspberrypi.com/documentation/accessories/camera.html#libcamera-vid):

```julia
3280×2464 @ 15 fps
1640×1232 @ 40 fps
1640×922 @ 45 fps
1920×1080 @ 45 fps
1280×720 @ 45 fps
480×640 @ 100-300 fps

```

My implementation is pretty basic (let me know if you see any possible improvements), but seems to work:

```julia
using ImageCore

function create_buffer(w, h)
    w2 = 64ceil(Int, w/64) # dimension adjustments to hardware restrictions
    h2 = 32ceil(Int, h/32)
    nb = Int(w2*h2*3//2) # total number of bytes per frame
    return (w2, h2, Vector{UInt8}(undef, nb))
end

struct Camera
    o::Base.Process
    buff::Vector{UInt8}
    img # a reshaped view into the bytes buffer 
    function Camera(w, h, fps)
        w2, h2, buff = create_buffer(w, h)
        b = view(buff, 1:w2*h2)
        Y = reshape(b, w2, h2)
        img = colorview(Gray, normedview(view(Y, 1:w, 1:h)))
        cmd = `libcamera-vid -n --framerate $fps --width $w --height $h -t 0 --codec yuv420 -o -` # I imagine that there might be a number of arguments improving things here, or tailoring it to what the user needs/wants
        o = open(cmd) # to "close" the camera just `kill(c.o)`
        new(o, buff, img)
    end
end

function read_frame(c::Camera)
    read!(c.o, c.buff) # not sure if `readbytes!` might be better here...
    return c.img
end

```

I got a lot of help from @therealdavidp, a friendly Raspberry Pi Engineer & Forum Moderator on the Raspberry Pi forum (see the actual post [here](https://forums.raspberrypi.com/viewtopic.php?p=1978205#p1978205)).

To me at least, this highlights the fact that both `VideoIO.jl` and `libcamera` can fetch frames from a camera, leaving the question of which is better suited for the job unclear.

I’m excited to see how testing this on the new Raspbian (ARMv8 64-bit) will fair…!