# Julia for live control of USB3.0 camera and DAC/ADC boards, with GUI?

**URL:** <https://discourse.julialang.org/t/julia-for-live-control-of-usb3-0-camera-and-dac-adc-boards-with-gui/14195>\
**Category:** General Usage\
**Created:** [August 28, 2018, 4:16pm UTC](https://discourse.julialang.org/t/julia-for-live-control-of-usb3-0-camera-and-dac-adc-boards-with-gui/14195 "2018-08-28T16:16:14Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![ianshmean](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ianshmean/32/216042_2.png) [@ianshmean](https://discourse.julialang.org/u/ianshmean)\
**Post date:** [August 28, 2018, 4:16pm UTC](https://discourse.julialang.org/t/julia-for-live-control-of-usb3-0-camera-and-dac-adc-boards-with-gui/14195/1 "2018-08-28T16:16:14Z")

</div>

I’ve prototyped a system in LabView that has the functionality I want, but I’m coming up against speed issues and would like to avoid developing further in a commercially licensed language.

The current labview system controls a USB3.0 camera, and records in raw AVI, while controlling a series of DAC/ADC boards in parallel. And it does it with a functional GUI interface. Basically it does what LabView does well… but I’m pushing it to its speed limits and dropping frames, and also want to move away from commercial software platforms for this system.

Would you say Julia is ready for this kind of thing? I mainly use it through IJulia, so real-time video-based GUIs seem pretty far from my experience. Are there any examples out there? What GUI packages are appropriate?

---

<div class="post-metadata">

**Author:** ![johnh](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/johnh/32/3615_2.png) [@johnh](https://discourse.julialang.org/u/johnh)\
**Post date:** [August 28, 2018, 5:10pm UTC](https://discourse.julialang.org/t/julia-for-live-control-of-usb3-0-camera-and-dac-adc-boards-with-gui/14195/2 "2018-08-28T17:10:54Z")

</div>

@ianshmean It would be great to help with this.  
VideoIO.jl comes to mind  
[https://github.com/kmsquire/VideoIO.jl](https://github.com/kmsquire/VideoIO.jl)  
I was going to say this is not being updated - but the latest changes were seven days ago, so it is looking better. Would VideoIO.jl read the images from your camera?  
I would think that a USB 3.0 device is probably pretty fast - can you just read an image from the device by opening and reading from a file?

I Cant say anything useful about GUI packages.

---

<div class="post-metadata">

**Author:** ![ianshmean](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ianshmean/32/216042_2.png) [@ianshmean](https://discourse.julialang.org/u/ianshmean)\
**Post date:** [August 28, 2018, 5:51pm UTC](https://discourse.julialang.org/t/julia-for-live-control-of-usb3-0-camera-and-dac-adc-boards-with-gui/14195/3 "2018-08-28T17:51:06Z")

</div>

The camera I’m trying to use is this: [https://www.ptgrey.com/grasshopper3-32-mp-mono-usb3-vision-sony-pregius-imx252](https://www.ptgrey.com/grasshopper3-32-mp-mono-usb3-vision-sony-pregius-imx252)

To read from it will probably require using ActiveX or directshow through the FlyCapture SDK  
[https://www.ptgrey.com/flycapture-sdk](https://www.ptgrey.com/flycapture-sdk)

I suspect that it’s a very long shot to expect Julia to handle this at _Julia_ speeds, because it would likely require Python interfacing, to get access to the camer

---

<div class="post-metadata">

**Author:** ![rdeits](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rdeits/32/286_2.png) [@rdeits](https://discourse.julialang.org/u/rdeits)\
**Post date:** [August 28, 2018, 5:54pm UTC](https://discourse.julialang.org/t/julia-for-live-control-of-usb3-0-camera-and-dac-adc-boards-with-gui/14195/4 "2018-08-28T17:54:19Z")

</div>

Perhaps you could use the flycapture C API from Julia? Calling C code from Julia is extremely efficient (essentially zero overhead) and quite easy to do: [https://docs.julialang.org/en/stable/manual/calling-c-and-fortran-code/index.html](https://docs.julialang.org/en/stable/manual/calling-c-and-fortran-code/index.html)

---

<div class="post-metadata">

**Author:** ![johnh](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/johnh/32/3615_2.png) [@johnh](https://discourse.julialang.org/u/johnh)\
**Post date:** [August 28, 2018, 6:03pm UTC](https://discourse.julialang.org/t/julia-for-live-control-of-usb3-0-camera-and-dac-adc-boards-with-gui/14195/5 "2018-08-28T18:03:48Z")

</div>

Not that anyone is interested, but I do have a FireWire camera somewhere, complete with cables and a PCI board.  
Completely useless these days, but at one point I was interested in FireWire for high speed networking.

---

<div class="post-metadata">

**Author:** ![johnh](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/johnh/32/3615_2.png) [@johnh](https://discourse.julialang.org/u/johnh)\
**Post date:** [August 28, 2018, 6:06pm UTC](https://discourse.julialang.org/t/julia-for-live-control-of-usb3-0-camera-and-dac-adc-boards-with-gui/14195/6 "2018-08-28T18:06:24Z")

</div>

I don not have the skills but Genicam looks to be the thing [GenICam - Wikipedia](https://en.wikipedia.org/wiki/GenICam)  
Maybe there will be a Julia implementation one day.

> **[GitHub - genicam/harvesters: Image Acquisition Library for GenICam-based...](https://github.com/genicam/harvesters)**
>
> Image Acquisition Library for GenICam-based Machine Vision System - GitHub - genicam/harvesters: Image Acquisition Library for GenICam-based Machine Vision System

you could try calling this from Julia?

---

<div class="post-metadata">

**Author:** ![piever](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/piever/32/1815_2.png) [@piever](https://discourse.julialang.org/u/piever)\
**Post date:** [August 28, 2018, 6:27pm UTC](https://discourse.julialang.org/t/julia-for-live-control-of-usb3-0-camera-and-dac-adc-boards-with-gui/14195/7 "2018-08-28T18:27:17Z")

</div>

Sorry for the off-topic (in the sense of “non-Julia”) but there are open source fast alternatives for this tasks (acquiring videos, real time processing and communicating with boards). Most people at my institute use bonsai: [Bonsai — Kampff Lab](http://www.kampff-lab.org/bonsai) (it’s based on C#, Microsoft Reactive framework and OpenCV as far as I understand). I’m not sure how is the C# interoperability with Julia.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [August 28, 2018, 6:38pm UTC](https://discourse.julialang.org/t/julia-for-live-control-of-usb3-0-camera-and-dac-adc-boards-with-gui/14195/8 "2018-08-28T18:38:31Z")

</div>

It might also be interesting to look into Julia bindings for libuvc, which is cross-platform and BSD-licensed: [GitHub - libuvc/libuvc: a cross-platform library for USB video devices](https://github.com/ktossell/libuvc)

---

<div class="post-metadata">

**Author:** ![kevin.squire](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kevin.squire/32/62_2.png) [@kevin.squire](https://discourse.julialang.org/u/kevin.squire)\
**Post date:** [August 28, 2018, 6:41pm UTC](https://discourse.julialang.org/t/julia-for-live-control-of-usb3-0-camera-and-dac-adc-boards-with-gui/14195/9 "2018-08-28T18:41:08Z")

</div>

Hi,

Can you use ffmpeg to get video from the camera? Some googling suggests people have tried with other Point Grey cameras before, although I didn’t look enough to know if they succeeded.

Anyway, my reason for asking is that VideoIO wraps ffmpeg, so if you can use ffmpeg to get video directly, then there’s a reasonable chance you can get VideoIO to read from it. (Note that, at the moment, VideoIO requires an older version of ffmpeg (\<= 3.0) to work).

Cheers,

Kevin

---

<div class="post-metadata">

**Author:** ![ianshmean](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ianshmean/32/216042_2.png) [@ianshmean](https://discourse.julialang.org/u/ianshmean)\
**Post date:** [August 28, 2018, 9:26pm UTC](https://discourse.julialang.org/t/julia-for-live-control-of-usb3-0-camera-and-dac-adc-boards-with-gui/14195/10 "2018-08-28T21:26:15Z")

</div>

**EDIT: I just realised misunderstood what they were sharing… I thought it was the code for how they interact with ffmpeg… but it’s just the ffmpeg source code… but at least it shows you can use ffmpeg!**

Interesting idea. PointGrey actually provide the ffmpeg source code used in their SDKs.

They require you to log in with a free account to access the file, so perhaps I shouldn’t provide the full download link here, but this is the text found through [https://www.ptgrey.com/support/downloads:](https://www.ptgrey.com/support/downloads:)

* * *

FFMPEG source code used in FlyCapture2 — 11/21/2017 - 13.660MB  
The FlyCapture2 and Spinnaker software uses code of FFmpeg [http://ffmpeg.org](http://ffmpeg.org) licensed under the LGPL v2.1 ([GNU Lesser General Public License v2.1 - GNU Project - Free Software Foundation](http://www.gnu.org/licenses/old-licenses/lgpl-2.1.html)). The ffmpeg code can be found online at [GitHub - FFmpeg/FFmpeg at 5156578d1f486163d5b83f1d63246cd23d107933](https://github.com/FFmpeg/FFmpeg/tree/5156578d1f486163d5b83f1d63246cd23d107933) or as attached.

* * *

It would be so great to have PointGrey camera support in VideoIO!

---

<div class="post-metadata">

**Author:** ![samuelpowell](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/samuelpowell/32/248_2.png) [@samuelpowell](https://discourse.julialang.org/u/samuelpowell)\
**Post date:** [August 28, 2018, 10:07pm UTC](https://discourse.julialang.org/t/julia-for-live-control-of-usb3-0-camera-and-dac-adc-boards-with-gui/14195/11 "2018-08-28T22:07:51Z")

</div>

Note that PointGrey/FLIR are migrating to their Spinnaker SDK for new cameras - so this might be a better target for any significant effort. The library is cross platform and there are C bindings available.

Regarding your original question I’m sure this is viable. I’ve previously used Julia to do some real time digital holography - grabbing frames from a Hamamatsu ORCA over FireWire (using a hacked fork of DC1394.jl) doing some FFTs and linear algebra, then displaying it all in real time on a Cairo canvas.

(The trickiest bit was making the rendering fast enough… I’m not sure how you’d manage that in, e.g., a notebook environment)

---

<div class="post-metadata">

**Author:** ![ianshmean](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ianshmean/32/216042_2.png) [@ianshmean](https://discourse.julialang.org/u/ianshmean)\
**Post date:** [August 28, 2018, 10:34pm UTC](https://discourse.julialang.org/t/julia-for-live-control-of-usb3-0-camera-and-dac-adc-boards-with-gui/14195/12 "2018-08-28T22:34:50Z")

</div>

Great to hear! And you’re right, the Spinnaker SDK would be the better target.  
If I get anywhere I’ll share progress…

Also, you’re right that the speed of the rendering live video in a GUI is a big challenge… especially if you want to serve it to a browser-based web app… you have to figure out a way to transcode down the raw result to a preview-appropriate resolution and framerate on the fly.

Is there anything like this out there in Julia? Perhaps for IJulia? Again, it’s one of those things that LabView does very well…

---

<div class="post-metadata">

**Author:** ![stemann](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stemann/32/4030_2.png) [@stemann](https://discourse.julialang.org/u/stemann)\
**Post date:** [October 30, 2018, 2:16pm UTC](https://discourse.julialang.org/t/julia-for-live-control-of-usb3-0-camera-and-dac-adc-boards-with-gui/14195/13 "2018-10-30T14:16:49Z")

</div>

There was this topic [Fastest way to blit images to a window](https://discourse.julialang.org/t/fastest-way-to-blit-images-to-a-window/1406) - mentioning an implementation of a Vimba wrapper in Julia.

Any news regarding wrapping Spinnaker? I am looking into wrapping Basler’s Pylon API and providing a generic Cameras (ImageAcquisition) package that should provide an AbstractCamera type.

---

<div class="post-metadata">

**Author:** ![samuelpowell](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/samuelpowell/32/248_2.png) [@samuelpowell](https://discourse.julialang.org/u/samuelpowell)\
**Post date:** [November 26, 2018, 6:41pm UTC](https://discourse.julialang.org/t/julia-for-live-control-of-usb3-0-camera-and-dac-adc-boards-with-gui/14195/14 "2018-11-26T18:41:44Z")

</div>

I’ve wrapped the Spinnaker SDK and provided a high level interface: [GitHub - samuelpowell/Spinnaker.jl: A Julia interface to the FLIR/PointGrey Spinnaker SDK](https://github.com/samuelpowell/Spinnaker.jl)

Certainly work in progress, but makes it very easy to set camera parameters and grab images from, e.g., Blackfly S, using Julia.

---

<div class="post-metadata">

**Author:** ![ianshmean](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ianshmean/32/216042_2.png) [@ianshmean](https://discourse.julialang.org/u/ianshmean)\
**Post date:** [November 26, 2018, 7:49pm UTC](https://discourse.julialang.org/t/julia-for-live-control-of-usb3-0-camera-and-dac-adc-boards-with-gui/14195/15 "2018-11-26T19:49:27Z")

</div>

Fantastic! I can confirm this works with the Grasshopper 3 USB3.0 camera on Windows 10  
I’ll continue to test and file any issues that arise.

Really awesome to have this functionality, thanks @samuelpowell

 ![42%20pm](https://global.discourse-cdn.com/julialang/original/3X/4/c/4c88924d6aa0b6e07cd05bfd8734bba349e26425.png)

---

<div class="post-metadata">

**Author:** ![stemann](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stemann/32/4030_2.png) [@stemann](https://discourse.julialang.org/u/stemann)\
**Post date:** [November 27, 2018, 2:21pm UTC](https://discourse.julialang.org/t/julia-for-live-control-of-usb3-0-camera-and-dac-adc-boards-with-gui/14195/16 "2018-11-27T14:21:19Z")

</div>

Great work!

Here is our current state of a (very minimal) high-level camera interface: [GitHub - IHPSystems/Cameras.jl: A Julia interface for cameras](https://github.com/IHPSystems/Cameras.jl)

It could be great if we could implement a common interface. See [[WIP] Cameras.jl](https://discourse.julialang.org/t/wip-cameras-jl/18067)

---

<div class="post-metadata">

**Author:** ![samuelpowell](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/samuelpowell/32/248_2.png) [@samuelpowell](https://discourse.julialang.org/u/samuelpowell)\
**Post date:** [November 27, 2018, 5:18pm UTC](https://discourse.julialang.org/t/julia-for-live-control-of-usb3-0-camera-and-dac-adc-boards-with-gui/14195/17 "2018-11-27T17:18:36Z")

</div>

A desirable outcome if viable. I’ll keep an eye on your package and see how it develops.

---

<div class="post-metadata">

**Author:** ![stemann](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stemann/32/4030_2.png) [@stemann](https://discourse.julialang.org/u/stemann)\
**Post date:** [November 27, 2018, 6:08pm UTC](https://discourse.julialang.org/t/julia-for-live-control-of-usb3-0-camera-and-dac-adc-boards-with-gui/14195/18 "2018-11-27T18:08:04Z")

</div>

OK. Feel free to comment/contribute. Have you considered naming property setters with a `set_`-prefix? E.g. `set_gain!` rather than `gain!` ?

---

<div class="post-metadata">

**Author:** ![samuelpowell](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/samuelpowell/32/248_2.png) [@samuelpowell](https://discourse.julialang.org/u/samuelpowell)\
**Post date:** [November 27, 2018, 6:18pm UTC](https://discourse.julialang.org/t/julia-for-live-control-of-usb3-0-camera-and-dac-adc-boards-with-gui/14195/19 "2018-11-27T18:18:48Z")

</div>

Feels more Julian to me to consider these as mutating the state of the camera, and hence `gain!(cam, setting)` in my implementation (and `gain(cam)` to retrieve it).

---

<div class="post-metadata">

**Author:** ![lab1104](https://avatars.discourse-cdn.com/v4/letter/l/77aa72/32.png) [@lab1104](https://discourse.julialang.org/u/lab1104)\
**Post date:** [September 9, 2019, 4:24pm UTC](https://discourse.julialang.org/t/julia-for-live-control-of-usb3-0-camera-and-dac-adc-boards-with-gui/14195/20 "2019-09-09T16:24:18Z")

</div>

Hi ianshmean,  
Sorry for out of topic question.  
I am also using FLIR Grasshopper USB3 camera, It works fine with Spinnaker software provided on thr site.  
I am trying to build a python code using Tkinter for GUI to capture live streaming of this camera. unfortunately nothing is working out…  
I am coding Jetson nano developer kit for this. Its Ubuntu 18.04.  
Have been struggling from more than 2months. Please if you could help