Hi, I have been following tutorials to make the switch from Python&Matlab to Julia. And one reminder that pops up all the time is to use loops in Julia since they are really fast. I was wondering why exactly this is the case? Vectorizing the code feels intuitively faster, so I am really curious why loops have such a great performance in Julia.
Thanks for helping me out!
To quote Chris,
It’s not that vectorization is fast
It’s that loops in Matlab/Python are slow.
I think the reason has something to do w type checks, & the Jit compiler…
Btw, Julia’s jit compiler uses the same architecture as Pythons Numba, so they achieve loops @ same speed.
Key difference is numba can only compile a strict subset of Python, very simple loops…
Vectorized code is expanded into loops anyway, in any language. That feeling comes from the fact that the loops are executed in lower level languages in R, Matlab, and Python.
In addition, one may be confused with loop vectorization, which is an optimization that takes advantage of the fact that processors can perform multiple operations at the same time. Some vectorized code may use that, and some explicit loops may need that to be set by the programer (see LoopVectorizations.jl).
Another possibility is that a “vectorized” operation is just a call to some sophisticated function that is very optimized for some task. Then you are not comparing simply to a loop. For example you won’t easily write three loops to do a matrix multiplication and get the performance of the functions that Matlab, Python, and Julia call when you do a matrix multiplication with A*B. But that is not simply about loops being fast or not.
On the other hand
@tullio a[i] = b[i,j]*c[j]
is incredibly fast and easy to understand for one line of code 
(see the thread)
Mostly don’t write loops, let macros do that for you!
Tullio is great, but I do not agree that this is a good advice in general. I would suggest better learning how to write performant loops. That gives us a flexibility that no macro can provide.
No doubt you should learn to write loops, but every opportunity you can turn 10 lines of code into 1 line of code with better or equal performance you should take it. Not everything is a tensor contraction or whatever, so loops are still a thing to learn.
Previously:
Understanding why loops are fast in Julia first requires you to understand why loops are slow in languages like Python and Matlab. In addition to @mbauman’s great post about this, I gave a course lecture about performance in dynamic languages based off of these notes.
Vectorizing the code feels intuitively faster, so I am really curious why loops have such a great performance in Julia.
Obviously, “vectorized” routines in Python and Matlab had to be implemented in terms of a loop somewhere, just not in Python or Matlab at the lowest level. That being said, there are two ways of stating this intuition, one of which is useful and one of which is pernicious:
Useful advice: Re-using someone else’s highly optimized library function is often a good idea. Why re-invent the wheel? (Also, they might know better algorithms/optimizations than you.)
Pernicious myth: Library code is fast, user code is slow. (Mere mortals can never match the performance achieved by the ancient ones!) This misleading “rule of thumb” is mainly due to languages like Python that do not allow you to write fast inner loops, so that “built-in” functions (or libraries calling low-level code from other languages) are privileged.
Even for the first point, there is a tension — although re-using high-quality libraries is good, optimized user code that is specialized for your problem can often beat the performance of a composition of more general-purpose building blocks, perhaps because you can combine multiple steps into one, or because you can omit computations that are not needed in your particular case, or because you can use a better data structure for your problem than the one forced on you by the library.
For a more ELI5 version of the excellent explanations above, I submit this simplistic, horrific generalization that also happens to be surprisingly accurate:
When something computationally intensive runs fast in Python, it’s actually running in C (or Fortran or …).
Vectorization is basically a tricky way of formulating a calculation in Python but outsourcing all the actual work to a language that can actually get stuff done. No need for that in Julia.
Thanks for all the reactions everyone! This really helped me out.