(A potentially naive question) On measuring Operational Intensity with LIKWID

Tooling Profiling
,
1

Hi,

I am somewhat new to Julia, and not a programming expert, but I been having great fun implementing a solver in it. I am getting close to the point where I need/want to profile it in more detail, and I am taking a look into LIKWID.jl (which seems great).

However, playing around, I noticed something I was not expecting. I created a small function with "tun

profiling_test.jl (859 Bytes)

able operational complexity":

function kernel(A, B, C,n)
     @inbounds  for i in eachindex(A)
         for j=1:n
            A[i] += B[i] * C[i] 
         end
    end
end

and I (maybe naively) hoped that the operational complexity would scale with n. Instead, what I got is

n=0001 MFLOPS=  749 OI=0.059 OI/n=0.059 data volume=0.3376
n=0002 MFLOPS=  748 OI=0.120 OI/n=0.060 data volume=0.3336
n=0003 MFLOPS= 1845 OI=0.179 OI/n=0.060 data volume=0.3360
n=0004 MFLOPS= 1249 OI=0.236 OI/n=0.059 data volume=0.3396
n=0005 MFLOPS= 1565 OI=0.296 OI/n=0.059 data volume=0.3376
n=0010 MFLOPS= 1131 OI=0.542 OI/n=0.054 data volume=0.3690
n=0020 MFLOPS=  948 OI=0.856 OI/n=0.043 data volume=0.4670
n=0050 MFLOPS= 1006 OI=1.660 OI/n=0.033 data volume=0.6024
n=0100 MFLOPS=  883 OI=2.097 OI/n=0.021 data volume=0.9539

that is, OI is roughly linear up to n=10, but then, apparently, the total transferred memory (data volume, in GB, in the above) starts to increase, and the linear scaling is lost. The full test file is attached.

So my question is: is this expected?

Thanks in advance for any help!!!

Eduardo Martini

profiling_test.jl (845 Bytes)

2

This indicates that the runtime is increasing, in proportion to volume. That could mean it’s just measuring background memory traffic. Also note that your MFLOPS is not increasing in n, meaning you are likely latency bound (instead of memory bound). You could try

a = A[i]; bc = B[i] * C[i]
for j = 1:n
    a += bc
end
A[i] = a
3

Yeah, I guess that is the case. I re ran the file now and got a much more consistent OI measurements, probably because whatever was creating the background traffic had ended.

As a curiosity: Is that expected, I figured that LIKWID would measure only the traffic related to the program being measured, not overall traffic. It clearly measures the MFLOP count correctly (on a separate test, it scales perfectly with n, and with my back of the envelope estimate), so there does not seem to be any contamination there. Why memory would be harder to measure?

4

Your original loop may happen to retrieve from memory in each j round. Because it might happen that A and B or C are aliased (i.e. reference the same data). I’m not sure about the status of the compiler on such things. You could try something like:


function kernel(A, B, C,n)
    B = Base.Experimental.Const(B)
    C = Base.Experimental.Const(C)
    Base.Experimental.@aliasscope begin
        @inbounds  for i in eachindex(A)
            for j=1:n
                A[i] += B[i] * C[i]
            end
        end
    end
end
5

I get basically the same thing with your function:

n=0001 MFLOP= 20.000 MFLOPS= 580 OI=0.062 OI/n=0.062 data volume=0.3239 vectorization= 0.00
n=0002 MFLOP= 30.000 MFLOPS= 744 OI=0.092 OI/n=0.046 data volume=0.3261 vectorization= 0.00
n=0003 MFLOP= 40.000 MFLOPS= 883 OI=0.121 OI/n=0.040 data volume=0.3307 vectorization= 0.00
n=0004 MFLOP= 50.000 MFLOPS= 920 OI=0.152 OI/n=0.038 data volume=0.3292 vectorization= 0.00
n=0005 MFLOP= 60.000 MFLOPS= 1017 OI=0.182 OI/n=0.036 data volume=0.3297 vectorization= 0.00
n=0010 MFLOP= 110.000 MFLOPS= 1431 OI=0.339 OI/n=0.034 data volume=0.3247 vectorization= 0.00
n=0020 MFLOP= 210.000 MFLOPS= 1365 OI=0.628 OI/n=0.031 data volume=0.3345 vectorization= 0.00
n=0050 MFLOP= 510.000 MFLOPS= 1131 OI=1.450 OI/n=0.029 data volume=0.3517 vectorization= 0.00
n=0100 MFLOP= 1010.000 MFLOPS= 920 OI=2.061 OI/n=0.021 data volume=0.4900 vectorization= 0.00

against the original version

n=0001 MFLOP= 20.000 MFLOPS= 410 OI=0.059 OI/n=0.059 data volume=0.3367 vectorization= 0.00
n=0002 MFLOP= 40.000 MFLOPS= 571 OI=0.122 OI/n=0.061 data volume=0.3267 vectorization= 0.00
n=0003 MFLOP= 60.000 MFLOPS= 774 OI=0.185 OI/n=0.062 data volume=0.3241 vectorization= 0.00
n=0004 MFLOP= 80.000 MFLOPS= 974 OI=0.247 OI/n=0.062 data volume=0.3243 vectorization= 0.00
n=0005 MFLOP= 100.000 MFLOPS= 857 OI=0.304 OI/n=0.061 data volume=0.3289 vectorization= 0.00
n=0010 MFLOP= 200.000 MFLOPS= 820 OI=0.420 OI/n=0.042 data volume=0.4765 vectorization= 0.00
n=0020 MFLOP= 400.000 MFLOPS= 799 OI=0.080 OI/n=0.004 data volume=4.9717 vectorization= 0.00
n=0050 MFLOP= 1000.000 MFLOPS= 756 OI=2.394 OI/n=0.048 data volume=0.4176 vectorization= 0.00
n=0100 MFLOP= 2000.000 MFLOPS= 756 OI=3.886 OI/n=0.039 data volume=0.5146 vectorization= 0.00

I have random spikes in memory movement, which is consistent with the background traffic hypothesis. I also turned off garbage collection to check, but that did not change anything.

I also don’t think the arrays are aliased, I am constructing them as

# Allocate large arrays

N = 10_000_000

A, B, C = rand(N), rand(N), rand(N)

and their entries have all different values…

(Proper profiling is harder than I thought :stuck_out_tongue: )

6

I think the inner loop is also preventing vectorisation, swapping them should help (or get rid of the for j=1:n loop entirely)

7

Yes, the loop order prevents vectorization, but that was on purpose. For large arrays, inverting loop order means that the arrays need to be reloaded on each j loop, thus increasing the memory transfer, and keeping OI constant. The original code (in theory) would prevent this data movement (the j loop could happen entirely in registry), thus effectively increasing OI.

For reference here are the two outputs:

@inbounds for i in eachindex(A)
for j = 1:n
A[i] += B[i] * C[i]
end
end

gives

n=0001 MFLOP=   20.000 MFLOPS=  364 OI=0.060 OI/n=0.060 data volume=0.3334 vectorization= 0.00 runtime=0.0549
n=0002 MFLOP=   40.000 MFLOPS=  637 OI=0.121 OI/n=0.060 data volume=0.3310 vectorization= 0.00 runtime=0.0628
n=0003 MFLOP=   60.000 MFLOPS=  855 OI=0.176 OI/n=0.059 data volume=0.3401 vectorization= 0.00 runtime=0.0702
n=0004 MFLOP=   80.000 MFLOPS=  967 OI=0.240 OI/n=0.060 data volume=0.3336 vectorization= 0.00 runtime=0.0827
n=0005 MFLOP=  100.000 MFLOPS=  879 OI=0.299 OI/n=0.060 data volume=0.3348 vectorization= 0.00 runtime=0.1137
n=0010 MFLOP=  200.000 MFLOPS=  831 OI=0.572 OI/n=0.057 data volume=0.3499 vectorization= 0.00 runtime=0.2407
n=0020 MFLOP=  400.000 MFLOPS=  793 OI=1.050 OI/n=0.052 data volume=0.3811 vectorization= 0.00 runtime=0.5045
n=0050 MFLOP= 1000.000 MFLOPS=  760 OI=2.161 OI/n=0.043 data volume=0.4627 vectorization= 0.00 runtime=1.3162
n=0100 MFLOP= 2000.000 MFLOPS=  744 OI=0.543 OI/n=0.005 data volume=3.6838 vectorization= 0.00 runtime=2.6882

while

for j = 1:n
@inbounds for i in eachindex(A)
A[i] += B[i] * C[i]
end
end

gives

n=0001 MFLOP=   20.000 MFLOPS=  868 OI=0.062 OI/n=0.062 data volume=0.3228 vectorization=100.00 runtime=0.0230
n=0002 MFLOP=   40.000 MFLOPS=  912 OI=0.062 OI/n=0.031 data volume=0.6437 vectorization=100.00 runtime=0.0438
n=0003 MFLOP=   60.000 MFLOPS=  898 OI=0.047 OI/n=0.016 data volume=1.2662 vectorization=100.00 runtime=0.0668
n=0004 MFLOP=   80.000 MFLOPS=  947 OI=0.062 OI/n=0.016 data volume=1.2863 vectorization=100.00 runtime=0.0845
n=0005 MFLOP=  100.000 MFLOPS=  937 OI=0.062 OI/n=0.012 data volume=1.6029 vectorization=100.00 runtime=0.1067
n=0010 MFLOP=  200.000 MFLOPS=  967 OI=0.063 OI/n=0.006 data volume=3.1638 vectorization=100.00 runtime=0.2069
n=0020 MFLOP=  400.000 MFLOPS=  962 OI=0.064 OI/n=0.003 data volume=6.2924 vectorization=100.00 runtime=0.4158
n=0050 MFLOP= 1000.000 MFLOPS=  978 OI=0.064 OI/n=0.001 data volume=15.6501 vectorization=100.00 runtime=1.0228
n=0100 MFLOP= 2000.000 MFLOPS=  971 OI=0.065 OI/n=0.001 data volume=30.9093 vectorization=100.00 runtime=2.0607

So using j in the outter loop increases the data movement linearly with n, whereas using j in the inner loop keeps the memory movement roughly equal (again with random spikes).

This does makes me think that my original code, by not using the memory bandwidth, does not leave room for the processor to do other stuff in the mean time, which gets tracked as bandwidth here. Using the j as outer loop clearly saturates the memory bandwidth, which seems to give the expected result.