# Fastest way to blit images to a window
**URL:** https://discourse.julialang.org/t/fastest-way-to-blit-images-to-a-window/1406
**Category:** Visualization
**Created:** [January 10, 2017, 8:51pm UTC](https://discourse.julialang.org/t/fastest-way-to-blit-images-to-a-window/1406 "2017-01-10T20:51:56Z")
**Posts on this page:** 17
**Page:** 1
### Author:  [@jtravs](https://discourse.julialang.org/u/jtravs)
#### Post date: [January 10, 2017, 8:51pm UTC](https://discourse.julialang.org/t/fastest-way-to-blit-images-to-a-window/1406/1 "2017-01-10T20:51:56Z")
Hi, I’m in the process of writing a laser beam profiler software with Julia. Fo this I am grabbing greyscale images from a scientific camera, that can be up to 4000x3000 pixels, stored as wither UInt8 or UInt16.
I now want to show these on the screen as fast as possible. So far I have tried Plots. jl using:
```julia
using Plots, Vimba # Vimba is the camera library (will be released as a package soon)
glvisualize()
# this returns a closure get() that grabs the next image
get, close = make_get()
plt = heatmap(get())
gui(plt)
# get() calls the c library using @threadcall so can be used async
function myrun()
while true
heatmap!(plt, get())
gui(plt)
end
end
@async myrun()
```
But even using glvisualize this is not super fast, definitely showing the images more slowly than my camera.
I have a rough code working in python using [GitHub - pyqtgraph/pyqtgraph: Fast data visualization and GUI tools for scientific / engineering applications](https://github.com/pyqtgraph/pyqtgraph) which is much faster, but that uses the Qt scenegraph, so I thought glvisualize might be faster. And I would much prefer to use Julia.
So my question is: what is the fastest way to show a colormapped image of a large 2d UInt8 or UInt16 array? I’m happy to use any graphics backend necessary. (As long as compatible with 64 bit Windows 10)
I have found this discussion [https://groups.google.com/d/topic/julia-users/rshpENHvHao/discussion](https://groups.google.com/d/topic/julia-users/rshpENHvHao/discussion), so could try Gtk.jl as suggested by @tim.holy but wondered if maybe GLVisualize code from @sdanisch might be better.
In the end I will also need some controls (sliders, numeric input, buttons) and also some simple line plots too.
Thanks for any help!
---
### Author:  [@tim.holy](https://discourse.julialang.org/u/tim.holy)
#### Post date: [January 10, 2017, 10:38pm UTC](https://discourse.julialang.org/t/fastest-way-to-blit-images-to-a-window/1406/2 "2017-01-10T22:38:48Z")
I would expect that GLVisualize to be the fastest, but that’s what Plots should be using (though perhaps there might be extra overhead). If you want to try Gtk, perhaps the easiest approach would be to use GtkUtilities.jl, where you can just `copy!` the image onto a `Canvas`. Example:
```julia
using GtkUtilities, Gtk.ShortNames, Colors
win = @Window()
c = @Canvas()
push!(win, c)
showall(win)
fill!(c, colorant"red")
# you should have a red window now
using Images, TestImages
img = testimage("mandrill");
copy!(c, img)
# you should see the mandrill now
```
---
### Author:  [@sdanisch](https://discourse.julialang.org/u/sdanisch)
#### Post date: [January 11, 2017, 12:24am UTC](https://discourse.julialang.org/t/fastest-way-to-blit-images-to-a-window/1406/3 "2017-01-11T00:24:40Z")
In such a case, I wouldn’t be surprised if Plots.jl is the bottleneck.
Also, is it possible to not allocate a new image every frame?
e.g. using `get!(buffer)`, or will this happen internally anyways?
try this code:
```nohighlight
using GLVisualize, Colors
w = glscreen()
function main(w, img)
while isopen(w)
rand!(img)# get[!](img)
GLAbstraction.set_arg!(obj, :image, reinterpret(Gray{U8}, img))
GLWindow.render_frame(w)
GLWindow.swapbuffers(w)
GLWindow.poll_glfw()
yield()
end
end
img = rand(UInt8, 4000, 4000)
imgi = reinterpret(Gray{U8}, img)
obj = visualize(imgi); _view(obj)
main(w, img)
GLWindow.destroy!(w)
```
This runs very smooth on my PC. If not, please open an issue!
Sliders, buttons etc can be brought in with GLPlot/GLVisualize!
E.g like in: [https://github.com/JuliaGL/GLVisualize.jl/blob/master/examples/gui/buttons.jl](https://github.com/JuliaGL/GLVisualize.jl/blob/master/examples/gui/buttons.jl)
You can also use Signals with a much easier setup (without GLWindow.render/swapbuffers etc) which I believe will almost be as fast, but to be sure I gave you the version with as little overhead as possible 😉
---
### Author:  [@sdanisch](https://discourse.julialang.org/u/sdanisch)
#### Post date: [January 11, 2017, 12:32am UTC](https://discourse.julialang.org/t/fastest-way-to-blit-images-to-a-window/1406/4 "2017-01-11T00:32:03Z")
> [@jtravs](#):
>
> colormapped image
Overlooked that!
It’s very similar, but will only work with floating point for now, so you’d need something like that:
```Julia
using GLVisualize, Colors, GeometryTypes
w = glscreen()
function main(w, img)
while isopen(w)
rand!(img)# get[!](img)
GLAbstraction.set_arg!(obj, :intensity, map(GLVisualize.Intensity{1, Float32}, img))
GLWindow.render_frame(w)
GLWindow.swapbuffers(w)
GLWindow.poll_glfw()
yield()
end
end
img = rand(UInt8, 4000, 4000)
imgi = map(GLVisualize.Intensity{1, Float32}, img)
obj = visualize(
imgi,
color_map = [RGBA{Float32}(1, 0, 0), RGBA{Float32}(0, 1, 0)], # any vector of colors will do
color_norm = Vec2f0(0, 255) # map intensity between 0 and 255 befor color lookup in [0-1]
)
_view(obj)
main(w, img)
GLWindow.destroy!(w)
```
Intensity only works for Float32 right now… If that introduces an overhead for you, please open an issue.
Should be straight forward to fix it to work with UInt8!
---
### Author:  [@sdanisch](https://discourse.julialang.org/u/sdanisch)
#### Post date: [January 11, 2017, 12:37am UTC](https://discourse.julialang.org/t/fastest-way-to-blit-images-to-a-window/1406/5 "2017-01-11T00:37:57Z")
It actually just seems to be this line, that doesn’t work with arbitrary element types:
[https://github.com/JuliaGL/GLVisualize.jl/blob/master/src/visualize/image\_like.jl#L41](https://github.com/JuliaGL/GLVisualize.jl/blob/master/src/visualize/image_like.jl#L41)
If you’re supplying the value for `color_norm` so that it calls `const_lift(extrema2f0, main)` it might just work.
---
### Author:  [@jtravs](https://discourse.julialang.org/u/jtravs)
#### Post date: [January 11, 2017, 10:19am UTC](https://discourse.julialang.org/t/fastest-way-to-blit-images-to-a-window/1406/6 "2017-01-11T10:19:51Z")
@sdanisch Awesome! That works extremely well! Thank you very much.
I need to somehow work this into a proper interface, so I’ll probably need more help at some point, but this displays the images perfectly.
One question: is there any documentation for things like `visualize` and its various arguments? I can’t find much and this seems to be quite a magic function!
Also, I’ll look through all the examples and code etc. but can you point me in the right direction for getting things like mouse dragging and scroll wheel working so that I can do panning etc. of the display. (I will work it out, but I just need some examples of catching the events etc.)
Thanks for an amazing package!
---
### Author:  [@sdanisch](https://discourse.julialang.org/u/sdanisch)
#### Post date: [January 11, 2017, 11:02am UTC](https://discourse.julialang.org/t/fastest-way-to-blit-images-to-a-window/1406/7 "2017-01-11T11:02:24Z")
Great!
There is:
```julia
arg1 = [Intensity{1, Float32}(1) for i=1:2, j=1:2]
visualize(arg1) -> visualize command
GLVisualize.get_docs(arg1) -> keyword args for above command with some docs.
GLVisualize.all_docs() -> Displays all the possible visualize args with a short description of the resulting visualization
```
Also, try `Pkg.test("GLVisualize")`! It’ll walk you through the examples with some guidance.
You should be able to drag and zoom with ctrl pressed + left mousebutton or scroll wheel…I guess this should be documented somewhere more prominent!
---
### Author:  [@ssfrr](https://discourse.julialang.org/u/ssfrr)
#### Post date: [January 11, 2017, 2:53pm UTC](https://discourse.julialang.org/t/fastest-way-to-blit-images-to-a-window/1406/8 "2017-01-11T14:53:21Z")
the code you posted for showing the camera image is great for spectrogram viewing, except I’m often dealing with really wide or tall aspect ratios. Is there an easy way to tell `visualize` to scale the aspect ratio to the window?
---
### Author:  [@sdanisch](https://discourse.julialang.org/u/sdanisch)
#### Post date: [January 11, 2017, 2:58pm UTC](https://discourse.julialang.org/t/fastest-way-to-blit-images-to-a-window/1406/9 "2017-01-11T14:58:04Z")
Yes, with the `primitive = SimpleRectangle(...)` keyword argument:
```nohighlight
using GLVisualize, Colors, GeometryTypes
w = glscreen()
img = rand(UInt8, 1000, 4000) #<---- different ratio
imgi = map(GLVisualize.Intensity{1, Float32}, img)
obj = visualize(
imgi,
primitive = SimpleRectangle(0, 0, 500, 500),
color_map = [RGBA{Float32}(1, 0, 0), RGBA{Float32}(0, 1, 0)], # any vector of colors will do
color_norm = Vec2f0(0, 255) # map intensity between 0 and 255 befor color lookup in [0-1]
)
_view(obj)
main(w, img)
GLWindow.destroy!(w)
```
Admittedly that is a bit low level…
Also for surface, you control the same property with `ranges = ((0, 100), (0, 100))`, which is inconsistent!
---
### Author:  [@ssfrr](https://discourse.julialang.org/u/ssfrr)
#### Post date: [January 11, 2017, 3:09pm UTC](https://discourse.julialang.org/t/fastest-way-to-blit-images-to-a-window/1406/10 "2017-01-11T15:09:08Z")
Hmm, that doesn’t seem to quite work for me. I’m invoking visualize with:
```julia
obj = visualize(
reinterpret(Gray{Float32}, data),
primitive = SimpleRectangle(0, 0, 500, 500))
```
Where `data` is a 1406x257 array of `Float32`, and then updating with `GLAbstraction.set_arg!(obj, :image, reinterpret(Gray{Float32}, data))`. What I get is below, which is cutting off the top of the image and showing white space on the right side.
Screen Shot 2017-01-11 at 10.04.43 AM.png
---
### Author:  [@sdanisch](https://discourse.julialang.org/u/sdanisch)
#### Post date: [January 11, 2017, 3:19pm UTC](https://discourse.julialang.org/t/fastest-way-to-blit-images-to-a-window/1406/11 "2017-01-11T15:19:41Z")
Sorry, I should answer more carefully 😉
Of course for the window aspect ratio, you need to insert the window size 😛
so you need:
```nohighlight
primitive = SimpleRectangle(0, 0, widths(window)...)
```
and if you want it to stay there, use:
```nohighlight
_view(obj, camera = :fixed_pixel)
```
---
### Author:  [@ssfrr](https://discourse.julialang.org/u/ssfrr)
#### Post date: [January 11, 2017, 4:16pm UTC](https://discourse.julialang.org/t/fastest-way-to-blit-images-to-a-window/1406/12 "2017-01-11T16:16:38Z")
Awesome, that’s much better, thanks! My mental model of how the various GL\* things work together is pretty weak.
---
### Author:  [@yakir12](https://discourse.julialang.org/u/yakir12)
#### Post date: [May 25, 2017, 7:40pm UTC](https://discourse.julialang.org/t/fastest-way-to-blit-images-to-a-window/1406/13 "2017-05-25T19:40:26Z")
@ssfrr, btw, what frame rates did you finally achieve?
---
### Author:  [@ecx2093](https://discourse.julialang.org/u/ecx2093)
#### Post date: [May 26, 2017, 2:54pm UTC](https://discourse.julialang.org/t/fastest-way-to-blit-images-to-a-window/1406/14 "2017-05-26T14:54:56Z")
@jtravs have you made any progress with the Vimba library? I have been trying to wrap the Vimba library and some other camera APIs using Clang.jl but have been running into some problems. Would be great to see a julia package for Vimba!
---
### Author:  [@stemann](https://discourse.julialang.org/u/stemann)
#### Post date: [October 30, 2018, 2:21pm UTC](https://discourse.julialang.org/t/fastest-way-to-blit-images-to-a-window/1406/15 "2018-10-30T14:21:42Z")
@jtravs Any news regarding Vimba? I am looking into wrapping Basler’s Pylon API, and it would make sense to create a generic Cameras/ImageAcquisition package providing a generic AbstractCamera type.
---
### Author:  [@jtravs](https://discourse.julialang.org/u/jtravs)
#### Post date: [October 30, 2018, 5:08pm UTC](https://discourse.julialang.org/t/fastest-way-to-blit-images-to-a-window/1406/16 "2018-10-30T17:08:42Z")
I put it up here: [https://gist.github.com/jtravs/ac92ab6dd184aafca1909246e4377485](https://gist.github.com/jtravs/ac92ab6dd184aafca1909246e4377485)
It is for a very old Julia and hasn’t been tested in years, but it did work quite well previously.
I would very much like (and probably use) a common Vimba/Pylon interface for cameras. I don;t think I have much time to work on it at the moment though.
---
### Author:  [@stemann](https://discourse.julialang.org/u/stemann)
#### Post date: [November 27, 2018, 2:32pm UTC](https://discourse.julialang.org/t/fastest-way-to-blit-images-to-a-window/1406/17 "2018-11-27T14:32:35Z")
See [[WIP] Cameras.jl](https://discourse.julialang.org/t/wip-cameras-jl/18067)