Why Julia is so much slower than Python? Invitation to Euler Project

General Usage Performance
1

The initial title of this thread was “Why Julia is so much slower than Python? Invitation to Euler Project”. Due to findings of Gerhard Aigner, I think that it should be renamed to “The builtin digits function is slow on BigInt.” I will do it a bit later.

The original post content is shown below.

In this post I was told to go and solve problem on Project Euler before starting to criticize Julia. So, I did.

Today, I solved its 20th problem with Python and Julia and found out that the code in Julia executes more than 52 times slower than the corresponding code in Python.

Currently, my code can be found here.

Unfortunately, because of the Project Euler rules, I may not post it in any other place.

However, this problem is quite trivial: you need to find the sum of digits of factorial of 100. So, you can easily, register on Project Euler, enter your solution to the problem and get to the post.

So, why Julia is so much slower than Python?

You are invited to post your own, quiker solution of this problem to the Project Euler site.

At the end I would like to thank Gerhard Aigner and Tamas Gal for their constructive comments and inputs.

On the other side, the first place in the nomination for toxic and unuseful comments in this tread so far should go to xiaodai, the second — to Moritz Schauer, and the third — devided between Josua Grawitter and Zlatan Vasović. As I could see, the post of Moritz Schauer has been liked by Tamas Papp and DNF.

So, so far, in this thread had shown himself 2 users who wanted to discuss the topic and 6 users who just wanted to troll. Based on this, I can estimate that Julia community is toxic on 75%.

2

I’ve a straight forward solution that runs in 201ns for n=10 and 1.2μs for n=100. How far is this off python? (With straight forward I mean using the obvious solution withstring and separate parsing).

1 Like
3

On my old computer, the Python solution for n=100 took 182 microseconds, while the same solution in Julia took 9512 microseconds (not counting compilation time).

P.S. In my Julia solution, I just use standard Julia functions and piping between them inside of my custom function.

4

@gevis a side-note: you are doing the brute-force method in both of your implementations (factorise and sum up the digits). That’s not what Project Euler wants to teach you and I am pretty sure you can post that code here, it will hurt nobody :wink: and help you to start a discussion.
You should think about the problem and simplify it mathematically.

Btw. @gevis code in Python takes 182us and in Julia 9512us.

Anyways: without looking into it, I guess the main reason of slowness is due to the approach: converting to BigInt and then calling digits and factorial. In Python, everything is basically BigInt and I guess the factorials are cached (not sure about Julia, I know that there is a cache too).

1 Like
5

This is much more closer to your Python implementation (but still the naive brute-force method) and three times faster than Python (on my machine):

julia> using BenchmarkTools

julia> fds(n) = n |> factorial |> digits |> sum
fds (generic function with 1 method)

julia> n = BigInt(100)

julia> @btime fds($n)
  75.942 μs (1413 allocations: 30.76 KiB)
648
6

Thank you for your reply. On my computer, your function executes 136.400 μs (1413 allocations: 30.76 KiB), a bit less than Python’s 178 microseconds for the similar code. However, you took conversion to BigInt out of its scope.

1 Like
8

It’s a bit of a stretch to ask people to comment on code you don’t share here. People here are quite eager to help and will look up your program on Euler… but do you yourself want them to spend their time on a scavenger hunt for the code?

15 Likes
9

Thank you for taking your time to make so valuable contribution to the discussion in this thread.

10

The funny thing is that I didn’t think of digits at first and instead used string and apparently its way faster than using the built in digits function, as this allocates. Inspecting this I found that

julia> euler20(n) =  mapreduce(x->Int(x)-48,+,string(factorial(big(n))))
euler20 (generic function with 1 method)

julia> @btime euler20(100)
  1.190 μs (13 allocations: 544 bytes)
648
4 Likes
11

IT’s the kind of speed that is into the who cares section.

12

I’m not making an account for some walled garden just to see what sounds like a trivial piece of code

1 Like
13

I disagree. The digits function is built into the language, and if it is slower than casting its argument into string, splitting it into array, and then casting all elements of this array back into intergers, it raises the question about the soundness of the programming language design. So, those who do not care about it, actually do not care about wider acceptance of the language itself.

15

Since it turns I’m that post’s author, I just want to clarify that I neither “told” you to do anything, nor to try just any problem out there in the world, without any prior mathematical insight. Or to quote myself:

As @tamasgal has shown, you can do it in 1 or 2 lines. Point proved. :smile:
If you want a solution more performant than 0.1ms (let me stress that, faster than 0.1ms!!), you can search the internet for better algorithms. That’s how it’s usually done.

3 Likes
16

It was actually was the same code that I used to solve the problem. I have just used the longer form for defining a function.

17

What is this post about?

There is a function in Julia (digits) which is slower than something else to achieve the same for a special type in a special case in this same language? Ok.

Therefore: the soundness of the programming language design is in question?

OP: you may understand that it is not easy to follow what you are trying to say.

What is your claim? What is your question? What do you need from us?

4 Likes
18

How could he know if you don’t share your code? :wink:

In general, you’ll always find specific examples which might be faster in another language, although the implementation is seemingly identical. Note that high-level languages hide a lot of code behind the scenes, so does Python and it’s also true for Julia, which is more complex due to the multiple dispatch semantics (it’s sometimes tricky to understand what’s called for beginners). You likEly will hit an edge case when you compare just a few function calls without examining what’s happening behind the scenes. A dramatic example is summing up consecutive integer numbers starting from 1 in a for-loop in Julia, which LLVM will boil down to n(n+1)/2, resulting in constant execution time independent from n. Python will just sum them up one by one.

Julia is definitely much faster than Python, that’s a fact. It’s more obvious in real-world examples or at least in slightly more complex ones where these tiny islands merge to a whole planet.

1 Like
19

It isn’t ‘built-in’, it’s an ordinary function, all in native Julia code.

It isn’t. digits is way faster than that, it just needs a special implementation for BigInts, which are profoundly different from ordinary integers.

:roll_eyes:

5 Likes
20

Just to keep it here.

21

Btw. can you pick another example? You’ll see that Python-pure solutions will almost always be orders of magnitudes slower than the same approach transpilled to Julia.

…and don’t use numpy, you want to compare languages, right? :wink: Even with numpy, you’ll only come close to Julia but not outperform it due to Pythons overhead. Let alone the fact that you are tied to what’s available in numpy and cannot implement customisations; for that you need e.g. numba, which is LLVM JIT, just like Julia.

You see where goes? Still thinking that Python is faster?

1 Like