Plotting images on GPU with ArrayFire

Specific Domains GPU
1

I’m combining lots of images into a single one and then add custom ticks and labels to the final image using Plots.jl library, then save it on the disk. However, we have millions of images to be processed, and it becomes very difficult for the CPU to do this in a reasonable amount of time.
After doing a Google search, I found that the ArrayFire library performs array operations on the GPU. I tested the load_image() function of ArrayFire for reading 10k images on GPU and I observed a 3-4x increase in speed comparing to the same operation on CPU with Images.jl’s load() function.
Next, I hopelessly tried my AFArray with Plots.jl and it didn’t work. Then I thought that if I perform the iterative operations on the GPU and then convert the whole array to something that plot() accepts, I shouldn’t have a great loss of performance, and asked this question on stackoverflow. After getting acquainted with the reinterpret() function, I realized the conversion from AFArray to Array{RGB{Float32},2} takes a huge amount of time (over a minute). Therefore I’m here to ask a few questions:

  1. Is there a clean way to plot images and draw axes and ticks entirely on the GPU?
  2. Is there a faster way to convert AFArray to something that plot() function would accept?
  3. Is there any other library working on GPU that would accept AFArray?
2

I know that Makie.jl (GitHub - JuliaPlots/Makie.jl: High level plotting on the GPU.) is supposed to be a “plotting on the gpu”, package, and is slated or hoped to be the next ‘mainstream’ plotting package.

I am not sure, though, how it interacts with gpu arrays, or whether it allows you to read images from file onto the gpu, like ArrayFire does.

Perhaps @sdanisch is able to respond to this?

1 Like
3

I am aware of the Makie.jl library and tried to do the same process with it and if I remember correctly it doesn’t have the hackable ticks feature that Plots has. This guy asked about that but he didn’t get an answer for Makie.jl.

4

ArrayFire can plot images on the GPU, see https://github.com/JuliaGPU/ArrayFire.jl/blob/master/examples/mandelbrot.jl

1 Like
5

Should probably move the array to CPU memory first, and then convert to the type you need. Otherwise I suspect it is doing element by element copy from GPU to CPU.

1 Like
6

I guess you can hack the axis: http://juliaplots.org/MakieReferenceImages/gallery//test_10/index.html
@jules can say more about the MakieLayout version.
@piever just recently also complained about the heatmap default, and had a fix for it I think.

It would be possible to plot images directly from the GPU in GLMakie, but it’s currently not implemented.

takes a huge amount of time (over a minute)

This must be a bug, it should only take a couple of ms (edit: @anon43133060 theory sounds about right :wink: ).

Did you try ImageIO & PNGFiles for loading? I heard they are supposed to be faster!

load_image() function of ArrayFire for reading 10k images on GPU and I observed a 3-4x

I hardly believe, that this is due to putting things on the GPU - putting it on the GPU is likely doing even more work. Instead, it’s likely simply faster, because they’re using a faster image loading library (on the CPU). One should check what they’re using and why it’s faster!

1 Like
7

If you use MakieLayout (now included in AbstractPlotting) you have several ways of setting different ticks, outlined here http://makie.juliaplots.org/stable/makielayout/laxis.html#Modifying-ticks

1 Like
8

Thank you so much for your answer! Unfortunately, I underestimated the potential of the Makie.jl library and I deeply regret it. Today I think I’ll be translating all my code into this library.

It would be possible to plot images directly from the GPU in GLMakie

That would be something I happily use :slight_smile: Looking forward to it.

putting it on the GPU is likely doing even more work

they’re using a faster image loading library

Ahh I get it now, so it wasn’t my GPU doing the trick. Maybe I should look for other ways to improve my performance but first let me see if Makie.jl and ImageIO does any better job with plotting.