Hi all,
I heard from a friend that this is a great place to have conversations about Julia. I’ve been using Python for years but am intrigued by Julia.
Big picture question: Will I have to try really hard to get Julia to be fast, or can I write Julia code much like I wrote Python code and it will just be faster.
First of all welcome. There are some noteworthy differences, the most significant of which are covered in here
https://docs.julialang.org/en/v1/manual/noteworthy-differences/#Noteworthy-differences-from-Python
Other than this I would simply give it a try. Like any other language Julia must be used properly if you want peak performance. If you have any trouble, please feel free to open up a thread in this section of julia’s discourse forum.
If you’re used to writing high performance python (eg using numpy extensively), there will be a couple things that feel weird at first. The biggest thing is that loops are fast, and you generally want to write your functions to work on scalars and use Broadcasting to then vectorize.
That said, if you follow a few pretty simple rules for good performance (type stability, reducing allocations) you should be pretty happy with the results.
This is also helpful:
https://docs.julialang.org/en/v1/manual/performance-tips/
In general, it is much easier to write fast Julia code than fast Python code.
Perhaps you are a very good python programmer (using numpy, numba, pypy, cython, etc) and that answer becomes more complicated. If you are only using pure python, most likely anything you write in Julia will be faster even if you do not do things the proper way (to start avoid non-constant global variables, which is the first performance tip). If you do things the proper way, simple Julia code will be much faster than simple Python code, except for short scripts.
That said, if you really want specific advice, try to port some python code where performance is important to you to Julia and post it here if you have any difficulty. Yes, this forum is great for learning Julia, and programming in general (I speak for myself).
I also recommend taking a look at how is a proper Julia development workflow. I have my own notes, but you will find other advice helpful. Particularly, in Julia, when developing, you should not start a new REPL all the time.
Short answer is no, you won’t have to try really hard. Some code will run faster relatively easily. For example writing some loops with arithmetic and assigning array elements (arrays are more or less as easy as lists in python) will usually just run fast. Array comprehensions look nearly the same as python list comprehensions, same with iterators; and they are in general much faster.
That said, there are some issues you should be aware of for performance. People have already pointed to the page for performance tips. Many or most of them are fairly straightforward and you can learn to code habitually for performance. That is, you don’t have to bend over backward to get performance. Even though python is in general not performant, you can learn coding styles that are more performant than others. I think the story is more or less the same in Julia.
I posted my experience here: Julia's applicable context is getting narrower over time? - #5 by Satvik . In short, I was able to take Python code that we’d spent ~200 hours optimizing, and make a faster Julia version in less than a week, my first time using Julia.
Julia is fast by default, but also has much stronger profiling tools, which makes optimization easy, as well as much better multithreading, which enables many optimizations that just wouldn’t be possible in Python.
But also this can depend on what your algorithms are doing. If they are just doing linear algebra with large matrices/etc. then the programming language has very little to do with it. Julia can’t magically make code calling BLAS code in MKL faster.
usually you write python code, an example of 10 lines nanmean in julia that works on general <:AbstractFloat number (even user defined ones), and the 100 lines of code in C++ that is used in python (which is much faster than numpy’s nanmean) that is slower than Julia and only works on Float32/64)
Hi friend! Welcome to the Julia Discourse! I asked a similar question a while back that might be of use to you: Why Are Languages Like Python and MATLAB So Much Slower than Julia?
Basically, I was asking about what makes Julia faster - in general - compared to languages like Python or MATLAB. All the answers here were great!
Even compared to simple vectorized Numpy operations Julia 1.6 is often faster (numbers in the plot > 1):
Source code:
Welcome. It is good to see new person from the Python sphere, it is still the language that I know the best.
First of all Julia is quite similar to Python on the surface syntax level, but there are noticeable differences. At the same time there are very different when you go deep. My college, who is physicist, starts using Julia thinking that is Python that is fast and as he himself notice, it was wrong attitude. But, when I was talking to him about it last time, he still doesn’t get Julia mindset for programming.
Second, against what Python code you measure you speed? Do you base your program on pure Python or calls to libraries? In Python counting chars in 1 MB string by string_var.counts(“A”) (or something like that) will most probably outperform by far (10, 20, 30, more times?) any thing that you can write by hand using for loops. If you write good Julia code by hand for the same task, it will take below twice time of string_var.counts(“A”). I would suspect very similar time basing on the fact that Python libraries are careful written in C (or similar language), but performers at such peak is not easy things and you can find unexpected regressions in performance.
Writing good code for most part means that you follow already mentioned many times performance tips. Julia documentation is quite dense, but for my taste following this tips can be compared to using Python’s PEPs. I know that “make indent four spaces” is much easier to understand that “avoid containers with abstract type parameters”, but I believe in practices you can be quite well off just memorizing few basic cases. Like “avoid containers with abstract type parameters” → avoid Real, Complex, Abstract Float, etc., use Int32, Int64, Float32 and Float64.
There is one of very big differences between Python and Julia. When using Python you get some rules of the game, you follow them or you need to code in C. In Julia you probably dig deeper, look behind the curtain and under the hood. If you as good as Chris Elrod you can improve your code performance in some amazing ways.
I don’t know if this is an answer to correct question, but I hope that it will help. If you want to hear about it, I will try to help as much as I can. And where I can’t smarter people than me will most definitely could.
While I think most of the comments in this thread are correct, I think they can be a bit misleading.
This Discourse community is full of threads from new users who thought they could basically just write Python or Matlab code in Julia, make some syntax tweaks and have it be orders of magnitude faster.
This is not a very realistic expectation. Python and Matlab in particular are two languages that strongly encourage a programming styles that are antithetical to high performance compilers.
Julia can be incredibly fast while being easy to write and having syntax similar to Python, but Julia is a different language and you still need to learn to write Julia code to take advantage of its speed
You can write Julia code much like you wrote Python code, and it could be much faster (Julia code against native Python loops), or much slower (e.g. naive Julia against numpy). It is not really hard to get Julia to be fast, but it may take time to learn the proper Julian ways. As an example see this recent thread, where the community helps a novice to improve the speed of his code by a factor of 70.
Plain Julia is not in general slower than Numpy (when your code is type-stable), see my benchmark above.
naive is not the same as plain
I tested the most simple way in Julia for element-wise operations using dotted operators, e.g.
a = rand(1000)
b = rand(1000)
y = a ./ b
z = exp.(a)
This is nearly the same syntax as in Numpy (just adding dots for broadcasting and not writing np. in front) and is also imho the most naive way to do it in Julia.
The only operations that were slower in Julia than in Numpy for me were 2-element additions and multiplications for (Float64) array sizes 10,000 to 100,000 (smaller and larger arrays were faster in Julia) and divisions for array size 100,000, all other tested operations were faster in Julia than in Numpy.
I’m yet to see a naive Julia vs. numpy (when the task is not memory/BLAS bounded) where Julia is MUCH slower. Unless by naive you mean intentionally type unstable. Example please?
One could argue that the most naive way of doing this in Julia is:
julia> a = rand(1000); b = rand(1000);
julia> y = zeros(1000);
julia> @btime for i in 1:length(a)
y[i] = a[i] / b[i]
end
92.402 μs (4980 allocations: 93.45 KiB)
which is of course pure-Python-painfully slow. Although probably that is not the kind of error an experienced python-numpy user would do.
Just for completeness, the not-deliberately type unstable function is not that bad:
julia> function f(a,b)
y = zeros(length(a))
for i in 1:length(a)
y[i] = a[i]/b[i]
end
y
end
f (generic function with 1 method)
julia> @btime f($a,$b);
1.412 μs (1 allocation: 7.94 KiB)
julia> @btime (y = $a ./ $b);
665.148 ns (1 allocation: 7.94 KiB)
you’re benchmarking in global scope:
julia> function f!(y,a,b)
for i in 1:length(a)
y[i] = a[i] / b[i]
end
y
end
f! (generic function with 1 method)
julia> @btime f!(y, $a, $b)
1.133 μs (0 allocations: 0 bytes)