I have a piece of code as follows
for i=1:n
b = (M .+d*N)*v + (M*g.+ N*h)*a
end
b, v and a are n x 1 vectors; while M and N are n x n matrices. d, g and h are constants that do not change during the loop as well as M and N; the only things that change are v and a.
Does it make any difference in running time precomputing the sum of these matrices (M .+d * N and M * g.+ N*h) out of the for loop and storing them? How does it behaviors when the size of these matrices increases?
The title of this post is due to the fact that I tested both ways and couldn’t find much difference in running time. So does Julia automatically identify this pattern and precomputes such sums?
This is an optimization known as loop-invariant code motion (LICM), which as the name suggests tries to identify code that doesn’t change in the loop and move it out of the loop. Julia’s compiler (and LLVM) sometimes is able to perform this optimization, and newer versions of Julia get better at, but it’s not always reliably able to identify when it can apply it.
So generally it is a good idea to hoist the code out of the loop yourself, especially when it is a relatively expensive calculation.
BTW: Manual LICM in Julia has some interesting slides about how LICM can work in the language of the compiler.
See also Loops, allocations and helping the coder
6 Likes
Thank you very much for your reply. It explains why I haven’t seen any difference in running time, for Julia must have optimized it automatically. Thanks again!