Webcam webapp

Specific Domains Web Stack
,
1

I want to have a webpage function as a viewer for the webcam.

In Genie, this is how I can do it without writing anything to disk and very low delay:

using JpegTurbo, Observables, VideoIO

using GenieFramework

@genietools

const baseurl = "/frame"
const refresh = Observable(true)

# refresh the image in the browser every 0.1 seconds
Timer(1; interval = 0.1) do _
    refresh[] = !refresh[]
end

cam = opencamera()
img = Ref(read(cam))
const w = size(img[], 1)

# fetch fresh frames from the camera as quickly as possible
Threads.@spawn while isopen(cam)
    read!(cam, img[])
    sleep(0.01)
end

# avoid writing to disk
route("/frame") do
    respond(String(jpeg_encode(img[])), :jpg)
end

@app begin
    @out imageurl = baseurl
    @onchange refresh begin
        # add an (invalid) anchor to the imagepath in order to trigger a reload in the Quasar/Vue backend
        imageurl = string(baseurl, "#", Base.time_ns())
    end
end
function ui()
    [imageview(src=:imageurl, basic=true, style="max-width: $(w)px")]
end
@page("/", ui)
Server.up()

Is it possible to do the same in, say, Oxygen?

2

Hello @yakir12,

yes Oxygen is perfectly capable to deliver a similar result. However, the scope of Oxygen is smaller than that of Genie, there is no counterpart to Stipple, so we need to write the front end in HTML and JavaScript ourselves. Usually you would serve those files statically, but to have everything in one file, I provide it as a string within my Julia code.

using JpegTurbo, VideoIO, Oxygen

const cam = opencamera()
const img = Ref(read(cam))
const ui = """
<!DOCTYPE html><html>
  <head>
    <meta charset="utf-8" />
    <title>Oxygen App</title>
  </head>
  <body>
    <div>
        <img id="frame">
    </div>
  <script>
    frame = document.querySelector("#frame");

    async function loadImage() {
      res = await fetch("/frame");
      imgData = await res.blob();
      frame.src = URL.createObjectURL(imgData);
    }

    setInterval(() => {
        loadImage();
    }, 33);
  </script>
  </body>
</html>
"""

# fetch fresh frames from the webcam
Threads.@spawn while isopen(cam)
    read!(cam, img[])
    sleep(1/30)
end

# define routes
@get "/" function()
    return ui
end

@get "/frame" function()
    return img[] |> jpeg_encode |> String
end

# start Oxygen server, in blocking mode
serve()
close(cam)

A technical difference to your version is also that the above code is not using websockets, but HTTP requests. So there is a bit more network overhead in the Oxygen version. But it might not relevant compared to the payload size, I didn’t notice any difference.

BR Stefan

1 Like
3

This is pretty cool. So theoretically if one knew how to program in JavaScript, one could reproduce some of what Stipple gives us “for free”.

One point of improvement (for any future reader), is that in the asynchronous loop where we update the img container, there is no reason to wait 1/30 seconds – that actually introduces a lag in the video (for me at least). It’s much better to either have a sleep(0.001) for instance, or just a yield():

# fetch fresh frames from the webcam
Threads.@spawn while isopen(cam)
    read!(cam, img[])
    yield()
end
4

Yes, I think if what you want to do in the Browser can be covered with Stipple, it is the better solution (a similar package is GitHub - plotly/Dash.jl: Dash for Julia - A Julia interface to the Dash ecosystem for creating analytic web applications in Julia. No JavaScript required.), if you are not fluent in JavaScript or TypeScript. However, I didn’t use either in an actual project, yet.

If things get more complex (not “just” a dashboard application), you probably will need to write the frontend in HTML / CSS / JavaScript. Then for the backend Genie, Oxygen or plain HTTP.jl seem to be the best Julia options currently.

The part of reading the images from the webcam, depends on the default of the actual hardware, resp. its default framerate, I think. I have no lag with 1/30 on my desktop, but on my notebook. In the docs, they sleep dependent on the frame rate (Reading Videos · VideoIO.jl).

5

Thanks for the tips.

My main use case is using it as some form of GUI. The main logic is run on a raspberry pi and the user interacts with the pi via an ethernet cable.