I’ve prototyped a system in LabView that has the functionality I want, but I’m coming up against speed issues and would like to avoid developing further in a commercially licensed language.
The current labview system controls a USB3.0 camera, and records in raw AVI, while controlling a series of DAC/ADC boards in parallel. And it does it with a functional GUI interface. Basically it does what LabView does well… but I’m pushing it to its speed limits and dropping frames, and also want to move away from commercial software platforms for this system.
Would you say Julia is ready for this kind of thing? I mainly use it through IJulia, so real-time video-based GUIs seem pretty far from my experience. Are there any examples out there? What GUI packages are appropriate?
@ianshmean It would be great to help with this.
VideoIO.jl comes to mind https://github.com/kmsquire/VideoIO.jl
I was going to say this is not being updated - but the latest changes were seven days ago, so it is looking better. Would VideoIO.jl read the images from your camera?
I would think that a USB 3.0 device is probably pretty fast - can you just read an image from the device by opening and reading from a file?
I suspect that it’s a very long shot to expect Julia to handle this at Julia speeds, because it would likely require Python interfacing, to get access to the camer
Not that anyone is interested, but I do have a FireWire camera somewhere, complete with cables and a PCI board.
Completely useless these days, but at one point I was interested in FireWire for high speed networking.
Sorry for the off-topic (in the sense of “non-Julia”) but there are open source fast alternatives for this tasks (acquiring videos, real time processing and communicating with boards). Most people at my institute use bonsai: Bonsai — Kampff Lab (it’s based on C#, Microsoft Reactive framework and OpenCV as far as I understand). I’m not sure how is the C# interoperability with Julia.
Can you use ffmpeg to get video from the camera? Some googling suggests people have tried with other Point Grey cameras before, although I didn’t look enough to know if they succeeded.
Anyway, my reason for asking is that VideoIO wraps ffmpeg, so if you can use ffmpeg to get video directly, then there’s a reasonable chance you can get VideoIO to read from it. (Note that, at the moment, VideoIO requires an older version of ffmpeg (<= 3.0) to work).
EDIT: I just realised misunderstood what they were sharing… I thought it was the code for how they interact with ffmpeg… but it’s just the ffmpeg source code… but at least it shows you can use ffmpeg!
Interesting idea. PointGrey actually provide the ffmpeg source code used in their SDKs.
They require you to log in with a free account to access the file, so perhaps I shouldn’t provide the full download link here, but this is the text found through https://www.ptgrey.com/support/downloads:
Note that PointGrey/FLIR are migrating to their Spinnaker SDK for new cameras - so this might be a better target for any significant effort. The library is cross platform and there are C bindings available.
Regarding your original question I’m sure this is viable. I’ve previously used Julia to do some real time digital holography - grabbing frames from a Hamamatsu ORCA over FireWire (using a hacked fork of DC1394.jl) doing some FFTs and linear algebra, then displaying it all in real time on a Cairo canvas.
(The trickiest bit was making the rendering fast enough… I’m not sure how you’d manage that in, e.g., a notebook environment)
Great to hear! And you’re right, the Spinnaker SDK would be the better target.
If I get anywhere I’ll share progress…
Also, you’re right that the speed of the rendering live video in a GUI is a big challenge… especially if you want to serve it to a browser-based web app… you have to figure out a way to transcode down the raw result to a preview-appropriate resolution and framerate on the fly.
Is there anything like this out there in Julia? Perhaps for IJulia? Again, it’s one of those things that LabView does very well…
Any news regarding wrapping Spinnaker? I am looking into wrapping Basler’s Pylon API and providing a generic Cameras (ImageAcquisition) package that should provide an AbstractCamera type.
Feels more Julian to me to consider these as mutating the state of the camera, and hence gain!(cam, setting) in my implementation (and gain(cam) to retrieve it).
Hi ianshmean,
Sorry for out of topic question.
I am also using FLIR Grasshopper USB3 camera, It works fine with Spinnaker software provided on thr site.
I am trying to build a python code using Tkinter for GUI to capture live streaming of this camera. unfortunately nothing is working out…
I am coding Jetson nano developer kit for this. Its Ubuntu 18.04.
Have been struggling from more than 2months. Please if you could help
