I’m currently working on developing a program to control a Spatial Light Modulator (SLM), which is a lab equipment used in the field of Structured Light. Essentially, an SLM allows for the projection of interesting shapes onto a laser beam.
From the programming point of view, we simply need to know that an SLM is a small monitor attached to your PC, in which we need to display images.
For a program to effectively control an SLM, it should have the following functionalities:
Display a grayscale image in full screen on a second monitor, without any additional elements being shown.
Allow for dynamic changes to the displayed image.
I came across the slmpy package, which precisely fulfills these requirements. However, it is built in Python, utilizing the wxPython module.
While exploring GUI packages in Julia, I haven’t found an obvious candidate for this task. I’m familiar with ImageView.jl, but unfortunately, it is unusable for me due to a known problem with GTK.jl on Windows.
Do you have any suggestions on how I can accomplish this in Julia? Thanks!
I haven’t tried this myself, but if there isn’t a native tool, you could serve a very simple website with a websocket to a browser that updates the current image. Then just F11 the browser into full screen on your second monitor.
Edit: maybe @ndortega could shed light on whether Oxygen.jl might help if you go the local-web route?
I use Gtk4.jl for this very application, displaying a grayscale image to control an SLM. You can get a list of connected monitors and fullscreen a window on a particular one. We identify our SLM by the tiny physical size it reports – we have no other 12 mm tall monitors! A separate window has controls so that we can dynamically control the phase mask we send to the SLM. In the lab I work in we use a Linux computer for this, and it has worked very smoothly so far. We have to update the calibration (when we need to) using the proprietary software that came with the SLM, with a USB connection to a separate (Windows) computer. But for sending masks to the SLM it’s great to be able to have Julia generate and display images instead of generating a bunch of TIFF files and then picking them out of a directory.
At this point, Gtk4.jl is at feature parity with Gtk.jl, and because it does not suffer from that upstream GTK3 issue you linked to, Gtk4.jl works much better on Windows. I do notice some REPL slowness on Windows, but it’s not extreme. I actually notice roughly the same amount of lag when I use PyPlot.jl on Windows.
using Gtk4
monitor_list = Gtk4.monitors() # list of monitors
isSLM(m) = (m.height_mm < 25) # in our case, this works. Other properties may be
# populated for your SLM, like manufacturer or model
i = findfirst(isSLM, monitor_list)
if i === nothing
error("SLM could not be found")
end
slm = monitor_list[i]
We create a window with a widget to display the image, and then call Gtk4.G_.fullscreen_on_monitor(window, slm) to fullscreen a GtkWindow on that monitor. That works on Linux at least, not sure about Windows.
For display widgets, there are many options:
• GtkPicture displays a GdkPixbuf image, which you can construct from an array.
• GtkDrawingArea uses Cairo to draw arbitrary images. This is what’s used in ImageView.jl, and it works exactly the same in Gtk4.jl and Gtk.jl.
• There is a separate package, Gtk4Makie, that can show a Makie figure in a Gtk4 window. You could use Makie’s image method to draw something using that. By the way, you might be able to avoid GTK altogether by using GLMakie, which uses a different cross platform library (GLFW) to show windows. I am not sure if it’s possible in GLFW to control where the window gets fullscreened.
If you use GtkPicture, the code would be something like:
p = GtkPicture()
win = GtkWindow(p) # make a window with `p` as its only child
Gtk4.G_.fullscreen_on_monitor(window,slm) # this will be wrapped nicely in Gtk4.jl soon
m = Matrix{GdkPixbufLib.RGB}(undef,1920,1152)
pb = GdkPixbuf(m,false) # false here means no alpha channel
function change_mask(m) # could add other arguments to control what gets written
# fill in the matrix `m` here
Gtk4.pixbuf(p,pb) # need to call this to refresh the picture
end
change_mask(m)
I don’t have access to the code currently, so I may come back and edit this.
Thank you for your suggestions! I’m actually considering all of them.
Regarding the code you just sent, I wasn’t able to make it work properly. I’m able to display a random grayscale image in fullscreen, but it appears on my main monitor, instead of the secondary one.
Here is my adaptation of your code:
using Gtk4
monitor_list = Gtk4.monitors()
slm = monitor_list[2] #Second monitor. Changing 2 to 1 doesn't make a difference
p = GtkPicture()
win = GtkWindow(p)
Gtk4.G_.fullscreen_on_monitor(win,slm)
convert2rgb(x) = Gtk4.GdkPixbufLib.RGB(x,x,x) #Converts numbers to gray
m = convert2rgb.(rand(UInt8,size(slm)...)) #Random grayscale image
pb = Gtk4.GdkPixbuf(m,false)
Gtk4.pixbuf(p,pb)
Are there any mistakes or do you think it can be due to the Windows related issues that you mentioned?
Argh, I tried this on Windows today and, indeed, it fullscreens on the main monitor, not the one specified. To be fair, I guess there is sort of a warning in the documentation that this may not work: Gtk.Window.fullscreen_on_monitor.
This is definitely an issue with the GTK4 library, not Gtk4.jl, so GNOME / gtk · GitLab would be the right place. This definitely isn’t going to be fixed any time soon, so I would look into other solutions.
I don’t have too much to add here, you summarized it pretty well. I think the only disclaimer is that integrating websockets into your server at the moment isn’t as clean as it can be. You have to create a middleware function to intercept the incoming connection like this. I plan on adding support for websocket routes similar to how it works in FastApi & Sanic.
But with that said, I would say the main benefit of using a web server to do this would allow you to build an API on top of your core application so it can easily integrate with other systems.
Of course, this approach requires you to write Julia, Javascript, and HTML to accomplish the same task while going with GTK4.jl would be a 100% Julia solution. At the end of the day, it depends entirely on your goals because both approaches have their own tradeoffs
. Could you point me to it?