Experiments with LoopVectorization and convolutions

Marco_Lombardi
Wed, 27 Nov 2024 20:50:58 +0000
Performance

I have been doing some experiments with the great LoopVectorization for convolution, and I must admit that in spite of my efforts to manually refactor my code, I have been unable to even approach the speed that this package is giving.

Consider the following code:

using LoopVectorization, OffsetArrays, PrettyChairmarks

A = OffsetArray(rand(150, 130))
kernel = OffsetArray(rand(31,21))
out = zeros(32:151, 22:131)

function f0!(out, A, kernel)
    @turbo for J ∈ CartesianIndices(out)
        tmp = zero(eltype(out))
        for I ∈ CartesianIndices(kernel)
            @inbounds tmp += A[J-I] * kernel[I]
        end
        out[J] = tmp
    end
end

When I test the speed of f0!, I get

julia> @bs f0!($out, $A, $kernel) seconds=1
Chairmarks.Benchmark: 1192 samples with 1 evaluation.
 Range (min … max):  695.264 μs …  1.077 ms  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     780.096 μs              ┊ GC (median):    0.00%
 Time  (mean ± σ):   796.765 μs ± 83.091 μs  ┊ GC (mean ± σ):  0.00% ± 0.00%

  ▆   ▇   █▃  ▂▆▁▁ ▄█▂   ▃▃    ▃▁     ▂                      ▃
  █▇▅██▅▆▆█████████████▆▆██▅▆▆▆██▆▄▅▅▄█▄▅▁▁▄▁█▅▁▅▇▁▁▁▅▄▄▄▄▅▄▁█ █
  695 μs        Histogram: log(frequency) by time      1.07 ms <

 Memory estimate: 0.0 bytes, allocs estimate: 0.

After a few hours and headaches, I was able to write my best manually optimized code for convolutions:

function f1!(out, A, kernel)
    @simd for J ∈ CartesianIndices(out)
        @inbounds out[J] = zero(eltype(out))
    end
    for I ∈ CartesianIndices(kernel)
        @inbounds k = kernel[I]
        @simd ivdep for J ∈ CartesianIndices(out)
            @inbounds out[J] = muladd(A[J-I], k, out[J])
        end
    end
end

However, it is at least a factor 3 slower than the LV version:

julia> @bs f1!($out, $A, $kernel) seconds=1
Chairmarks.Benchmark: 388 samples with 1 evaluation.
 Range (min … max):  2.255 ms …   4.595 ms  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     2.430 ms               ┊ GC (median):    0.00%
 Time  (mean ± σ):   2.429 ms ± 173.565 μs  ┊ GC (mean ± σ):  0.00% ± 0.00%

  ▆ █        ▄▂        ▅▆▁      ▁  ▂▁         ▆▄
  █▆█▆▁▁▄▇▇▆▇██▆██▇█▆▆▄███▆▁▇▁█▄█▆▄██▄▇█▇▄█▁█▇███▇▁▄▁▁▄▁▁▁▁▆▄ ▇
  2.26 ms      Histogram: log(frequency) by time      2.73 ms <

 Memory estimate: 0.0 bytes, allocs estimate: 0.

Note that this has nothing to do with loop unrolling, since @turbo unroll=-1 produces exactly the same results; I obtain instead even better execution times using @turbo unroll=4. Also, trying to use directly the SIMD package did not help in any way (I just reproduced the same speed of f1!).

What is LV doing under the hoods? Can I hope to obtain a similar speed?

25 posts - 10 participants