Very slow loading speed of TIFF images

New to Julia
1

I’m very new to Julia. I’m trying to replace my Matlab code with Julia. I compared TIFF image loading speeds in Matlab and Julia, but the Matlab’s imread() is way faster than the Images’ load(). Is there any way to speed up the TIFF image loading?

The followings are codes I used. The test image was a 565 x 552 RGB image (~2MB).

Matlab

fname = "test_image.tiff";
tic;
for n = 1:5
    img = double(imread(fname));
end
toc;

> Elapsed time is 0.149297 seconds.
> Elapsed time is 0.039244 seconds.
> Elapsed time is 0.040094 seconds.

Julia

using Images
fname =  "test_image.tiff"
juliaRead() = begin
	for n ∈ 1:5
		load(fname)
	end
end
@time juliaRead()

>  3.442524 seconds (1.83 M allocations: 137.901 MiB, 37.48% gc time, 33.34% compilation time: 55% of which was recompilation)
>  1.193470 seconds (2.29 k allocations: 20.828 MiB, 98.60% gc time)
>  1.196542 seconds (2.29 k allocations: 20.828 MiB, 98.45% gc time)
2

Whay package are you using to read tiff images?

3

mkitti,

I tried the Images package which I believed it was using the TiffImages function.

4

With GMT (a 601x601x3 geotiff image)

julia> using GMT
julia> @btime gdalread("hawaii_south.tiff");
  16.711 ms (101 allocations: 1.05 MiB)

and for comparison

julia> using Images
julia> @btime load("hawaii_south.tiff");
  189.968 ms (199612 allocations: 34.36 MiB)

But Matlab is the clear winner.

>> tic; for n = 1:5, img = imread('hawaii_south.tiff'); end; toc
Elapsed time is 0.015118 seconds.

Note that I dropped the double(...) to make the times comparable and note also that Matlab time is multiplied by 5. If we divide by it we get 0.0030236

5

Note that the original post’s timings are almost completely garbage collection. I wonder why loading five images incurs a second worth of garbage, though.

6

I guess you are using windows operating system? On windows, before loading the image, load firstly calls GC.gc() to recycle memory. This is a known issue. See TiffImages.jl/load.jl at master · tlnagy/TiffImages.jl · GitHub
and the safe_load in TiffImages.jl/utils.jl at master · tlnagy/TiffImages.jl · GitHub
On Linux system, there is no such issue and the loading time is approximately the same as the Matlab. I test with a 10MB image.

7

The variation in times is really big. Especially a 100x for Matlab. This is suspicious. Most likely some deferred processing is taking place. Perhaps the file is simply memory mapped and nothing else.

Have you tried:

using TiffImages
img = TiffImages.load(filepath; mmap=true);
1 Like
8

Am I reading that code right? To work around a problem with mmapping on Windows, a preemptive GC.gc is called regardless of lazy/mmap flags and with no way to opt out?

9

That’s right. I’m using windows operating system. Thank you for your comments. Since almost all machines in my environment are Windows, that would be great if there is a way to get the same speed on Windows. If you know any way to speed up, that would be very helpful.

10

Thank you very much!
I also tried GMT as you’ve shown. This speeds up a lot.

using GMT
readGMT() = begin
	for n ∈ 1:5
		GMT.gdalread(filetif)
	end
end
@time readGMT()
  0.095000 seconds (680 allocations: 10.446 MiB)

At this moment, the solution would be the usage of the GMT package.

Thanks!

11

There is one update.
I’ve tried using OpenCV.jl. It’s seems the best option I can take on my Windows10 machine.
I summarized the runtime comparison what I’ve tried so far. The test image is a 565 x 552 RGB image (~2MB).

Package time
OpenCV.jl 3.6 ms
GMT.jl 14.5 ms
TiffImages.jl 305.4 ms
Matlab 8.0 ms

Versions

  • julia v1.9.0
  • Matlab R2022a
  • OpenCV v4.5.3
  • TiffImages v0.6.4
  • GMT v1.1.0

Codes

julia

using OpenCV
using TiffImages
using GMT

filetif = "test_image.tiff"

@btime OpenCV.imread(filetif)
    3.631 ms (20 allocations: 720 bytes)
@btime TiffImages.load(filetif)
    305.342 ms (413 allocations: 4.16 MiB)
@btime GMT.gdalread(filetif)
    14.536 ms (141 allocations: 2.09 MiB)

Matlab

filetif = "test_image.tiff"

tic;
for i = 1:1000
    img = imread(filetif);
end
toc;
    Elapsed time is 7.980724 seconds for 1000 images
     -> 7.980724 ms for each.
12

@Kadii I love that you tried using OpenCV.jl, it’s great that OpenCV.jl can read it the fastest but then it must be noticed that OpenCV.jl doesn’t work really well with other image processing libraries in Julia that are provided by JuliaImages. So despite saving time on imread, you are forced to stay in OpenCV’s world atm mostly in OpenCV.jl.

13

@Ashwani_Rathee Thank you for your advise. I’ve been faced the problem you mentioned.
I realized that I could convert OpenCV::Mat to ::Array by simply calling its slices. I haven’t tried yet but I’m hoping I can pass it to JuliaImages’ functions and it works. :slight_smile:

using OpenCV

filetif = "test_image.tiff"
img = OpenCV.imread(filetif, -1)
arr = img[:, :, :]

typeof(img)
    OpenCV.Mat{UInt8}

typeof(arr)
    Array{UInt8, 3}
14

How can load() from Images.jl be improved?

15

I’ve worked around the GC issue like so:

"""
    tif_load(path)

Loads a TIFF image after temporarily disabling garbage collection,
then re-sets GC to its initial state. This is a workaround for
the 75-90% GC overhead otherwise seen when loading TIFF images.
"""
function tif_load(path)
    initial_gc_state = GC.enable(false)
    img = load(path)
    GC.enable(initial_gc_state)
    img
end

which does a pretty good job:

julia> @time TiffImages.load("test.tif"); # test.tif is a 532 x 532 RGB image
  0.193545 seconds (298 allocations: 1.513 MiB, 98.69% gc time)

julia> @time tif_load("test.tif");
  0.001620 seconds (298 allocations: 1.513 MiB)
2 Likes
16

Author of TiffImages here, I consider it a bug if we’re slower than Matlab :sweat_smile:

Are you also on Windows, I presume? Does your tif_load function work with the mmap=true flag set too?

I wonder if we still need that GC.load call at all (or if there’s a better solution), which was introduced in https://github.com/tlnagy/TiffImages.jl/pull/79. Unfortunately, I don’t regularly use a Windows machine so this performance gotcha has evaded me.