Very different performance in different computers due to garbage collection?

TheLeache
Wed, 21 Sep 2022 10:00:41 +0000
Performance

So, I have a code which calculates a measure of “openness” or “sprawl” around points in a series of big rasters from the National Land Cover Dataset. I had used this code on my personal laptop (A 2020 ROG G14 with a Ryzen 4800HS) and the whole thing ran more or less in one night for all years that this dataset is available.

I had to run it again with a small correction, so I ran it on my new desktop at my PhD office (Dell Prcision 3650 with Intel Xeon W-1350) and it took FOREVER to run. This was very strange, so I looked at this old benchmark that I made when I was deciding whether to run the code in Python or Julia:

using Images

function count_notocean(arr, mask)
    if size(mask) == masksiz
        return count(i -> i !=11, arr[maskindex])
    else
        tempindex = findall(i -> i == 0, mask)
        return count(i -> i !=11, arr[tempindex])
    end
end

function count_developed(arr, mask)
    if size(mask) == masksiz
        temp = arr[maskindex]
        return (2*count(i -> i == 21, temp) + 4*count(i -> i == 22, temp) + 6*count(i -> i == 23, temp) + 8*count(i -> i == 24, temp))>>4   #Approximation fully in integers
    else
        tempindex = findall(i -> i == 0, mask)
        temp = arr[tempindex]
    return (2*count(i -> i == 21, temp) + 4*count(i -> i == 22, temp) + 6*count(i -> i == 23, temp) + 8*count(i -> i == 24, temp))>>4   #Approximation fully in integers
    end
end

function point_sprawl(arr, i, j, mask)
    temprows = (max(i-siz,1), min(i+siz, size(arr)[1]))
    tempcols = (max(j-siz,1), min(j+siz, size(arr)[2]))
    temp = arr[temprows[1]:temprows[2], tempcols[1]:tempcols[2]]
    tempmask =  mask[(temprows[1]:temprows[2]).-i.+siz.+1, (tempcols[1]:tempcols[2]).-j.+siz.+1]
    developed = count_developed(temp, tempmask)
    notocean = count_notocean(temp, tempmask)
    undeveloped = notocean-developed
    spr = trunc(undeveloped/notocean*255)
    return spr
end

function calc_sprawl(arr, mask)
    spr = zeros(UInt8, size(arr)[1], size(arr)[2]).+255
    Threads.@threads for i in siz:(size(arr)[1]-siz)
        Threads.@threads for j in siz:(size(arr)[2]-siz)
            if arr[i,j] == 21 || arr[i,j] == 22 || arr[i,j] == 23 || arr[i,j] == 24
                spr[i,j] = point_sprawl(arr, i, j, mask)
            else
                nothing
            end
        end
    end
    return spr
end

cd("D:/thesis_suburb")

rs= load("code_replication/test/austin.png")
rs = reinterpret(UInt8, rs)
rst = copy(rs)

mask = load("code_replication/luts/circle_mask.png")
mask = reinterpret(UInt8, mask)
const maskindex = findall(i -> i == 0, mask)

const masksiz = size(mask)
const siz = Int(trunc(size(mask)[1]/2))

function main_mask()
    sprawl = calc_sprawl(rst, mask)
end

@time(sprawl_single = main_mask())

All this does is take a test image from the NLCD and compute the “sprawl” index for pixels which have codes for being built-up. The code I’m using is much larger but here’s where the performance differences are.

This is what I get from the laptop:

julia> @time(sprawl_single = main_mask())
  3.815631 seconds (4.84 M allocations: 6.051 GiB, 49.27% gc time)

And this is what I get from the desktop:

julia> @time(sprawl_single = main_mask())
 52.764730 seconds (4.84 M allocations: 6.051 GiB, 97.21% gc time)

This is running Julia 1.7.3 in both computers, both through the REPL through the VSCode extension, environment installations are a copy of one another through the manifest and files are mirrored through Syncthing. The laptop uses Windows 11 Pro, the desktop uses Windows 10 Pro for Workstations. Any clues as to why the same simple code might perform so differently in two different machines? Why is the garbage collector acting up on the desktop and taking so much time to perform its tasks?

29 posts - 7 participants