So, I have a code which calculates a measure of “openness” or “sprawl” around points in a series of big rasters from the National Land Cover Dataset. I had used this code on my personal laptop (A 2020 ROG G14 with a Ryzen 4800HS) and the whole thing ran more or less in one night for all years that this dataset is available.
I had to run it again with a small correction, so I ran it on my new desktop at my PhD office (Dell Prcision 3650 with Intel Xeon W-1350) and it took FOREVER to run. This was very strange, so I looked at this old benchmark that I made when I was deciding whether to run the code in Python or Julia:
using Images
function count_notocean(arr, mask)
if size(mask) == masksiz
return count(i -> i !=11, arr[maskindex])
else
tempindex = findall(i -> i == 0, mask)
return count(i -> i !=11, arr[tempindex])
end
end
function count_developed(arr, mask)
if size(mask) == masksiz
temp = arr[maskindex]
return (2*count(i -> i == 21, temp) + 4*count(i -> i == 22, temp) + 6*count(i -> i == 23, temp) + 8*count(i -> i == 24, temp))>>4 #Approximation fully in integers
else
tempindex = findall(i -> i == 0, mask)
temp = arr[tempindex]
return (2*count(i -> i == 21, temp) + 4*count(i -> i == 22, temp) + 6*count(i -> i == 23, temp) + 8*count(i -> i == 24, temp))>>4 #Approximation fully in integers
end
end
function point_sprawl(arr, i, j, mask)
temprows = (max(i-siz,1), min(i+siz, size(arr)[1]))
tempcols = (max(j-siz,1), min(j+siz, size(arr)[2]))
temp = arr[temprows[1]:temprows[2], tempcols[1]:tempcols[2]]
tempmask = mask[(temprows[1]:temprows[2]).-i.+siz.+1, (tempcols[1]:tempcols[2]).-j.+siz.+1]
developed = count_developed(temp, tempmask)
notocean = count_notocean(temp, tempmask)
undeveloped = notocean-developed
spr = trunc(undeveloped/notocean*255)
return spr
end
function calc_sprawl(arr, mask)
spr = zeros(UInt8, size(arr)[1], size(arr)[2]).+255
Threads.@threads for i in siz:(size(arr)[1]-siz)
Threads.@threads for j in siz:(size(arr)[2]-siz)
if arr[i,j] == 21 || arr[i,j] == 22 || arr[i,j] == 23 || arr[i,j] == 24
spr[i,j] = point_sprawl(arr, i, j, mask)
else
nothing
end
end
end
return spr
end
cd("D:/thesis_suburb")
rs= load("code_replication/test/austin.png")
rs = reinterpret(UInt8, rs)
rst = copy(rs)
mask = load("code_replication/luts/circle_mask.png")
mask = reinterpret(UInt8, mask)
const maskindex = findall(i -> i == 0, mask)
const masksiz = size(mask)
const siz = Int(trunc(size(mask)[1]/2))
function main_mask()
sprawl = calc_sprawl(rst, mask)
end
@time(sprawl_single = main_mask())
All this does is take a test image from the NLCD and compute the “sprawl” index for pixels which have codes for being built-up. The code I’m using is much larger but here’s where the performance differences are.
This is what I get from the laptop:
julia> @time(sprawl_single = main_mask())
3.815631 seconds (4.84 M allocations: 6.051 GiB, 49.27% gc time)
And this is what I get from the desktop:
julia> @time(sprawl_single = main_mask())
52.764730 seconds (4.84 M allocations: 6.051 GiB, 97.21% gc time)
This is running Julia 1.7.3 in both computers, both through the REPL through the VSCode extension, environment installations are a copy of one another through the manifest and files are mirrored through Syncthing. The laptop uses Windows 11 Pro, the desktop uses Windows 10 Pro for Workstations. Any clues as to why the same simple code might perform so differently in two different machines? Why is the garbage collector acting up on the desktop and taking so much time to perform its tasks?
