# real-time camera acquisition and processing, howto

**URL:** <https://discourse.julialang.org/t/real-time-camera-acquisition-and-processing-howto/20398>\
**Category:** Performance\
**Created:** [February 3, 2019, 1:38pm UTC](https://discourse.julialang.org/t/real-time-camera-acquisition-and-processing-howto/20398 "2019-02-03T13:38:22Z")\
**Posts on this page:** 1\
**Showing post:** 8

<div class="post-metadata">

**Author:** ![ianshmean](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ianshmean/32/216042_2.png) [@ianshmean](https://discourse.julialang.org/u/ianshmean)\
**Post date:** [February 4, 2019, 4:02pm UTC](https://discourse.julialang.org/t/real-time-camera-acquisition-and-processing-howto/20398/8 "2019-02-04T16:02:48Z")

</div>

> [@mkborregaard](#):
>
> I think the question is how to avoid the lag, so you can display the video in real time. I think either @ssfrr or @ianshmean were working with real-time video capture at some point?

Indeed! This is the `Makie` approach I arrived at, with dummy data generated by `rand!()` and an efficient downsampling approach for lower resolution previews.

Also there’s a preview scalefactor speed testing example below which demonstrates that on my setup downsampling is only beneficial at lower than a ~0.5 preview scale factor

```julia
using Random #For generating fake live video data
using Makie, ImageTransformations, Interpolations, Colors, FixedPointNumbers

nframes = 500
previewscalefactor = 0.2

img = rand(UInt8,1536,2048) #Raw image
scene = Scene(resolution = (size(img,2),size(img,1)),colormap=Reverse(:Greys)) #Reverse colormap to match normal greyscale image representation 

preview_size = (round(Int,size(img,1)*previewscalefactor), round(Int,size(img,2)*previewscalefactor))
#Lower resolution preview, with efficient downsampling
buff = zeros(N0f8, preview_size[1],preview_size[2])
itp = interpolate!(reinterpret(N0f8, img), BSpline(Linear()))
#Set up Makie scene for preview
hmap = heatmap!(scene, buff, show_axis = false, scale_plot = false)[end]
display(scene)

#Test dummy image capture with preview
t = @elapsed for i = 1:nframes
    rand!(img) # aquire new data into raw image array
    ImageTransformations.imresize!(buff, itp)
    hmap[1] = buff
    yield()
end
println("Video generation & live preview at ",previewscalefactor," scale: ",round(nframes/t,digits=1)," FPS")

```

Video generation & live preview at 0.2 scale: 286.0 FPS

**Testing preview scale factor**

```julia
using Random #For generating fake live video data
using Makie, ImageTransformations, Interpolations, Colors, FixedPointNumbers

nframes = 500
img = rand(UInt8,1536,2048) #Raw image

#First test dummy image capture only
t = @elapsed for i = 1:nframes
    rand!(img) # aquire new data into raw image array
end
println("Video generation: ",round(nframes/t,digits=1)," FPS")

for previewscalefactor in [1,0.5,0.25,0.1]
    scene = Scene(resolution = (size(img,2),size(img,1)),colormap=Reverse(:Greys)) #Reverse colormap to match normal greyscale image representation 
    if previewscalefactor != 1
        #Lower resolution preview, with efficient downsampling
        preview_size = (round(Int,size(img,1)*previewscalefactor), round(Int,size(img,2)*previewscalefactor))
        buff = zeros(N0f8, preview_size[1],preview_size[2])
        itp = interpolate!(reinterpret(N0f8, img), BSpline(Linear()))
        #Set up Makie scene for preview
        hmap = heatmap!(scene, buff, show_axis = false, scale_plot = false)[end]
    else
        hmap = heatmap!(scene, img, show_axis = false, scale_plot = false)[end]
    end

    display(scene)

    #Test dummy image capture with preview
    t = @elapsed for i = 1:nframes
        rand!(img) # aquire new data into raw image array
        if previewscalefactor !=1
            ImageTransformations.imresize!(buff, itp)
            hmap[1] = buff
        else
            hmap[1] = img
        end
        yield()
    end
    println("Video generation & live preview at ",previewscalefactor," scale: ",round(nframes/t,digits=1)," FPS")
end

```

Video generation: 2137.1 FPS  
Video generation & live preview at 1.0 scale: 67.0 FPS  
Video generation & live preview at 0.5 scale: 60.8 FPS  
Video generation & live preview at 0.25 scale: 207.1 FPS  
Video generation & live preview at 0.1 scale: 536.4 FPS

---

_[View the full topic](https://discourse.julialang.org/t/real-time-camera-acquisition-and-processing-howto/20398)._