A way to capture screenshots with Julia programatically

General Usage
1

Today I found a simple way to capture a screenshot of your desktop using Julia and thought I would share.

I guess it is a hidden feature of the ImageMagick.jl package.

Just run

using ImageMagick
screenshot = ImageMagick.load("screenshot:")

If you also load the Images package it will display as any other image would. Or you can also use FileIO to easily save it to a PNG or other format.

I tested this on Windows 10 and it works quite well. It has a few hiccups with high DPI screens.
If you have multiple monitors, you instead get a 3D array where the third axis selects the screen.

From some quick searching, it’s possible than “screenshot:” should be replaced with “x:root” on linux but I don’t have a linux install to test with.

6 Likes
2

Hmm, it’s not working on my ubuntu installation

julia> screenshot = ImageMagick.load("x:root")
ERROR: UnableToOpenBlob `x:root': No such file or directory @ error/blob.c/OpenBlob/2873
Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:33
 [2] error(wand::MagickWand)
   @ ImageMagick ~/.julia/packages/ImageMagick/mOSDB/src/libmagickwand.jl:187
 [3] readimage
   @ ~/.julia/packages/ImageMagick/mOSDB/src/libmagickwand.jl:274 [inlined]
 [4] load_(file::String, permute_horizontal::Bool; ImageType::Type, extraprop::String, extraprop
ertynames::Nothing, view::Bool)
   @ ImageMagick ~/.julia/packages/ImageMagick/mOSDB/src/ImageMagick.jl:146
 [5] load_
   @ ~/.julia/packages/ImageMagick/mOSDB/src/ImageMagick.jl:139 [inlined]
 [6] #load#30
   @ ~/.julia/packages/ImageMagick/mOSDB/src/ImageMagick.jl:126 [inlined]
 [7] load(::String)
   @ ImageMagick ~/.julia/packages/ImageMagick/mOSDB/src/ImageMagick.jl:126
 [8] top-level scope
   @ REPL[9]:1
3

That annoying! And I take it screenshot: didn’t work either?

4

No luck

julia> screenshot = ImageMagick.load("screenshot:")
ERROR:
Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:33
 [2] error(wand::MagickWand)
   @ ImageMagick ~/.julia/packages/ImageMagick/mOSDB/src/libmagickwand.jl:187
 [3] readimage
   @ ~/.julia/packages/ImageMagick/mOSDB/src/libmagickwand.jl:274 [inlined]
 [4] load_(file::String, permute_horizontal::Bool; ImageType::Type, extraprop::String, extraprop
ertynames::Nothing, view::Bool)
   @ ImageMagick ~/.julia/packages/ImageMagick/mOSDB/src/ImageMagick.jl:146
 [5] load_
   @ ~/.julia/packages/ImageMagick/mOSDB/src/ImageMagick.jl:139 [inlined]
 [6] #load#30
   @ ~/.julia/packages/ImageMagick/mOSDB/src/ImageMagick.jl:126 [inlined]
 [7] load(::String)
   @ ImageMagick ~/.julia/packages/ImageMagick/mOSDB/src/ImageMagick.jl:126
 [8] top-level scope
   @ REPL[13]:1
5

That’s too bad! I was hoping this would work cross-platform. Looking through the Image Magick C code there’s both a linux and Windows code path. Not sure why one is not supported.

Oh well, hopefully it can still be useful for Windows users.

6

You could use Python’s PIL.ImageGrab.grab() via PyCall.

Looking at the source code, it seems feasible to put together a Julia package with similar functionality.

(Note that the Win32 code is here in C, and looks relatively complicated but could be implemented via ccall. The X11 code is here. I wonder if there is a simpler way to do it, though?)

3 Likes
7

Thanks for the pointer @stevengj. The grabclipboard function could really come in handy too if it were ported.

8

On ubuntu 20, calling imagemagick from the command line worked for me as a workaround:

using FileIO
using ImageIO

cmd = `import -window root png:-`
io = read(cmd)
img = load(IOBuffer(io))
1 Like
9

I think this would helpful for linux people with the screen shots with framerate for bgra ones over 90 fps, rgb around 2fps and gray also 2fps(I could use help how to fix that) and offset of originx, origin y are available and width and height of screenshot are definable:

https://github.com/ashwani-rathee/GrabScreen.jl

I am still working on the performance issues with RGB and GRAY, X11 isn’t most flexible.

Issue is that X11 returns me with vector of uint8 with 4 bits for pixel, for let’s 100*100, it would be 40000 values, for BGRA ones I directly do

    res = reinterpret(BGRA{N0f8}, res)
    res = reshape(res , Width, Height)
    res =  res'

Which is pretty fast but then when I have to something similiar for RGB, Gray it becomes a issue
RGB.(res), Gray.(res) are quite slow for realtime purposes and I am not able to think of faster solution atm.

3 Likes