Hi everyone,
I am testing two parallel versions of my loop - one with @threads
and one with @spawn
. Weirdly, @spawn
finishes in a fraction of the time @threads
needs, makes much fewer allocations, and gives me this weird message that I cannot find an explanation for anywhere: Task (runnable) @0x00007efb776e3c70
The output of both options match (also with the non-parallelized one). Can anyone explain to me what is happening?
Two more side questions:
- Does anyone know how to get the functions to make less allocations? Everything that normally works returns errors ( using
.=
instead of=
or using@view
when accessingto_inter
) when using it withinterpolate
. - How can I set up
cache_m
in a less weird way? Trying to set it up as a matrix of the output type of interpolate somehow does not work.
Here is a MWE:
# Setup
using Interpolations, BenchmarkTools
ID = rand(1,3000)
to_inter = rand(100,100,3000)
# Setting up output matrix as a vector of interpolation elements (please excuse the weird way of setting it up)
cache_m = []
for i in eachindex(ID)
if i == 1
cache_m = [interpolate(to_inter[:,:,1], BSpline(Cubic(Line(OnGrid()))))]
else
cache_m = vcat(cache_m, [cache_m[1]])
end
end
cache_mt = cache_m
cache_ms = cache_m
# The actual functions. They only differ by their parallelization type. It computes interpolations for each ID.
function test_np!(to_inter, cache_m)
for i = eachindex(ID)
cache_m[i] = interpolate( to_inter[:,:,i], BSpline(Cubic(Line(OnGrid()))))
end
end
function test_pt!(to_inter, cache_mt)
Threads.@threads for i = eachindex(ID)
cache_mt[i] = interpolate( to_inter[:,:,i], BSpline(Cubic(Line(OnGrid()))))
end
end
function test_ps!(to_inter, cache_ms)
Threads.@spawn for i = eachindex(ID)
cache_ms[i] = interpolate( to_inter[:,:,i], BSpline(Cubic(Line(OnGrid()))))
end
end
# Run first time to trigger compilation
test_np!(to_inter, cache_m)
test_pt!(to_inter, cache_mt)
test_ps!(to_inter, cache_ms)
# Time functions
@time test_np!(to_inter, cache_m) # 1.763476 seconds (592.91 k allocations: 1.035 GiB, 4.53% gc time)
@time test_pt!(to_inter, cache_mt) # 0.128616 seconds (498.39 k allocations: 1.033 GiB)
@time test_ps!(to_inter, cache_ms) # 0.000577 seconds (192 allocations: 360.953 KiB) Task (runnable) @0x00007f8fb697c2f0
# Check if output is the same
cache_m == cache_mt # true
cache_m == cache_ms # true
Thanks a lot!