Hi, I’m in the process of writing a laser beam profiler software with Julia. Fo this I am grabbing greyscale images from a scientific camera, that can be up to 4000x3000 pixels, stored as wither UInt8 or UInt16.
I now want to show these on the screen as fast as possible. So far I have tried Plots. jl using:
using Plots, Vimba # Vimba is the camera library (will be released as a package soon)
glvisualize()
# this returns a closure get() that grabs the next image
get, close = make_get()
plt = heatmap(get())
gui(plt)
# get() calls the c library using @threadcall so can be used async
function myrun()
while true
heatmap!(plt, get())
gui(plt)
end
end
@async myrun()
So my question is: what is the fastest way to show a colormapped image of a large 2d UInt8 or UInt16 array? I’m happy to use any graphics backend necessary. (As long as compatible with 64 bit Windows 10)
I would expect that GLVisualize to be the fastest, but that’s what Plots should be using (though perhaps there might be extra overhead). If you want to try Gtk, perhaps the easiest approach would be to use GtkUtilities.jl, where you can just copy! the image onto a Canvas. Example:
using GtkUtilities, Gtk.ShortNames, Colors
win = @Window()
c = @Canvas()
push!(win, c)
showall(win)
fill!(c, colorant"red")
# you should have a red window now
using Images, TestImages
img = testimage("mandrill");
copy!(c, img)
# you should see the mandrill now
In such a case, I wouldn’t be surprised if Plots.jl is the bottleneck.
Also, is it possible to not allocate a new image every frame?
e.g. using get!(buffer), or will this happen internally anyways?
try this code:
using GLVisualize, Colors
w = glscreen()
function main(w, img)
while isopen(w)
rand!(img)# get[!](img)
GLAbstraction.set_arg!(obj, :image, reinterpret(Gray{U8}, img))
GLWindow.render_frame(w)
GLWindow.swapbuffers(w)
GLWindow.poll_glfw()
yield()
end
end
img = rand(UInt8, 4000, 4000)
imgi = reinterpret(Gray{U8}, img)
obj = visualize(imgi); _view(obj)
main(w, img)
GLWindow.destroy!(w)
You can also use Signals with a much easier setup (without GLWindow.render/swapbuffers etc) which I believe will almost be as fast, but to be sure I gave you the version with as little overhead as possible
Overlooked that!
It’s very similar, but will only work with floating point for now, so you’d need something like that:
using GLVisualize, Colors, GeometryTypes
w = glscreen()
function main(w, img)
while isopen(w)
rand!(img)# get[!](img)
GLAbstraction.set_arg!(obj, :intensity, map(GLVisualize.Intensity{1, Float32}, img))
GLWindow.render_frame(w)
GLWindow.swapbuffers(w)
GLWindow.poll_glfw()
yield()
end
end
img = rand(UInt8, 4000, 4000)
imgi = map(GLVisualize.Intensity{1, Float32}, img)
obj = visualize(
imgi,
color_map = [RGBA{Float32}(1, 0, 0), RGBA{Float32}(0, 1, 0)], # any vector of colors will do
color_norm = Vec2f0(0, 255) # map intensity between 0 and 255 befor color lookup in [0-1]
)
_view(obj)
main(w, img)
GLWindow.destroy!(w)
Intensity only works for Float32 right now… If that introduces an overhead for you, please open an issue.
Should be straight forward to fix it to work with UInt8!
@sdanisch Awesome! That works extremely well! Thank you very much.
I need to somehow work this into a proper interface, so I’ll probably need more help at some point, but this displays the images perfectly.
One question: is there any documentation for things like visualize and its various arguments? I can’t find much and this seems to be quite a magic function!
Also, I’ll look through all the examples and code etc. but can you point me in the right direction for getting things like mouse dragging and scroll wheel working so that I can do panning etc. of the display. (I will work it out, but I just need some examples of catching the events etc.)
arg1 = [Intensity{1, Float32}(1) for i=1:2, j=1:2]
visualize(arg1) -> visualize command
GLVisualize.get_docs(arg1) -> keyword args for above command with some docs.
GLVisualize.all_docs() -> Displays all the possible visualize args with a short description of the resulting visualization
Also, try Pkg.test("GLVisualize")! It’ll walk you through the examples with some guidance.
You should be able to drag and zoom with ctrl pressed + left mousebutton or scroll wheel…I guess this should be documented somewhere more prominent!
the code you posted for showing the camera image is great for spectrogram viewing, except I’m often dealing with really wide or tall aspect ratios. Is there an easy way to tell visualize to scale the aspect ratio to the window?
Where data is a 1406x257 array of Float32, and then updating with GLAbstraction.set_arg!(obj, :image, reinterpret(Gray{Float32}, data)). What I get is below, which is cutting off the top of the image and showing white space on the right side.
@jtravs have you made any progress with the Vimba library? I have been trying to wrap the Vimba library and some other camera APIs using Clang.jl but have been running into some problems. Would be great to see a julia package for Vimba!
@jtravs Any news regarding Vimba? I am looking into wrapping Basler’s Pylon API, and it would make sense to create a generic Cameras/ImageAcquisition package providing a generic AbstractCamera type.
It is for a very old Julia and hasn’t been tested in years, but it did work quite well previously.
I would very much like (and probably use) a common Vimba/Pylon interface for cameras. I don;t think I have much time to work on it at the moment though.
