Julia slower than Python to sort and reverse a list of integers

[lucasmsoares96]

My goal is to compare Julia’s performance against other languages like Python, Scala and Rust to perform some simple tasks. My first task was to sort an array of 999999 integers read from a text file.

The code below runs at a similar time in Python and in julia

import time

st = time.time()
f = open("random_numbers.txt", "r")
lines = f.readlines()
numbers = list(map(lambda x : int(x.strip()), lines))
numbers.sort()
numbers.reverse()
et = time.time()
elapsed_time = et - st
print('Execution time:', elapsed_time, 'seconds')

0.4491417407989502 seconds

import Base: parse

parse(x) = y -> parse(x,y)
@time begin
    lines = readlines("random_numbers.txt")
    (lines .|> parse(Int64)) |> sort |> reverse
end;

1.122733 seconds

But when I add a print in the codes there is a significant performance discrepancy between Julia and Python

import time

st = time.time()
f = open("random_numbers.txt", "r")
lines = f.readlines()
numbers = list(map(lambda x : int(x.strip()), lines))
numbers.sort()
numbers.reverse()
for n in numbers:
    print(n)
f.close() 
et = time.time()
elapsed_time = et - st
print('Execution time:', elapsed_time, 'seconds')

4.248031377792358 seconds

import Base: parse

parse(x) = y -> parse(x,y)
@time begin
    const lines = readlines("random_numbers.txt")
    (lines .|> parse(Int64)) |> sort |> reverse .|> println
end;

10.152690 seconds

Any hints as to what could be causing this underperformance of the Julia?

[jar1]

Check out the Performance Tips · The Julia Language

[giordano]

I can’t reproduce your timings. I get

  0.515662 seconds (4.16 M allocations: 125.246 MiB, 9.27% gc time, 15.40% compilation time)

for Julia, vs

Execution time: 0.7124240398406982 seconds

for Python. And I can halve the Julia runtime by simplifying the code to

@time let
    lines = readlines("random_numbers.txt")
    (lines .|> Base.Fix1(parse, Int64)) |> sort |> reverse
end

Now I get

  0.261280 seconds (2.00 M allocations: 86.157 MiB, 6.06% gc time)

For reference, my platform is

julia> versioninfo()
Julia Version 1.8.5
Commit 17cfb8e65ea (2023-01-08 06:45 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 8 × Intel(R) Core(TM) i7-4870HQ CPU @ 2.50GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-13.0.1 (ORCJIT, haswell)
  Threads: 1 on 8 virtual cores

[lucasmsoares96]

could you test the code version with println?

import Base: parse

parse(x) = y -> parse(x,y)
@time begin
    const lines = readlines("random_numbers.txt")
    (lines .|> parse(Int64)) |> sort |> reverse .|> println
end;

my platform is:

julia> versioninfo()
Julia Version 1.8.5
Commit 17cfb8e65ea (2023-01-08 06:45 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 8 × Intel(R) Core(TM) i7-4870HQ CPU @ 2.50GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-13.0.1 (ORCJIT, haswell)
  Threads: 4 on 8 virtual cores
Environment:
  JULIA_NUM_THREADS = 4

[giordano]

It’s all about 8 seconds for both Python and Julia, but I don’t know what you’re measuring at this point: reading from file? parsing? sorting? reversing? printing? This is getting all very confusing.

[lucasmsoares96]

8 seconds for both? There must be something wrong with my environment. My Julia spends 10 seconds while python spends 4 seconds.

The biggest discrepancy happens when I add the println in Julia

[jar1]

Julia 1.9-rc2 --startup-file=no:
16.069481 seconds (11.35 M allocations: 350.027 MiB, 1.09% gc time, 1.53% compilation time)

Python 3.10.8: Execution time: 5.241699695587158 seconds

I find this to be a practical benchmark that combines multiple functions in a realistic way, so the performance is worth evaluating.

[jar1]

The Python one is sorting and reversing the list in place.

In Julia that’s

import Base: parse

parse(x) = y -> parse(x,y)
@time begin
    const lines = readlines("random_numbers.txt")
    rows = [parse(Int, l) for l in lines]
    sort!(rows)
    reverse!(rows)
    println.(rows)
end;

but it makes no difference

15.431943 seconds (10.21 M allocations: 309.901 MiB, 1.10% gc time, 0.85% compilation time)

[jar1]

I think it’s that Python buffers stdout by default and Julia doesn’t.

Println performance bug in interactive terminal (I.e. much slower than c, perl, ruby, python)

slow printing in terminals

from

Why is printing to a terminal slow?

A different use of printing to stdout/stderr is to pipe output through less (or similar pager) to be able to inspect the output interactively without having to do a full (long) run. E.g. grep for certain strings in the output to check a condition, then letting the program continue until the next matching line, etc. I find this very useful during development. In this case you will also hit upon the slower output of Julia compared to Python. A simple (rough, due to the user keyboard interaction)…

[giordano]

This is why mixing up multiple things together isn’t helpful: the fact that printing might be slow isn’t surpring at all, and trying to optimise multiple things together when there’s a single significant bottleneck is a waste of time.

[jar1]

We can start with a program with a problem and then reduce from there.

[jar1]

With buffering it’s faster:

import Base: parse

parse(x) = y -> parse(x,y)
@time begin
    io = IOBuffer()
    const lines = readlines("random_numbers.txt")
    rows = [parse(Int, l) for l in lines]
    sort!(rows)
    reverse!(rows)
    println.(io, rows)
    write(stdout, take!(io))
end;

  1.774918 seconds (5.14 M allocations: 188.944 MiB, 9.57% gc time, 6.51% compilation time)

compared to 5.5 s in Python.

[lucasmsoares96]

Amazing!! Thank you very much!

[DNF]

@jar1

sort!(rows)
reverse!(rows)

It is more idiomatic to simply sort in reverse directly:

sort!(rows; rev=true)

[Mateusz_K]

Keep in mind that you are comparing Python’s TimSort implemented in C, against Julia’s default QuickSort. Which are not only different algorithms but also C+Python’s overhead is compared against pure Julia solution. Fair comparison would be implementing this algorithm in Python and than comparing. It’s just silly otherwise, because you basically start another program from Python to make a claim about python. Julia can also call C routines.

[tbeason]

I don’t think it is silly because these are the default sorting routines, which is what the vast majority of people will use.

[Mateusz_K]

What i am saying is that conclusion is false, regardless of what defaults people would use. It’s the same as calling C library form Julia in an attempt to benchmark Julia’s speed. That’s just dishonest benchmark regardless of what defaults are. Julia is good at certain things, so is Python but calling another software, leave alone another algorithm to say something about language is wrong. He could compare algorithms by C call from one of languages, if he has wanted to compare algorithms. He could compare implementation in these two languages if he wanted to compare languages. Julia has all sorts of sorting algorithms, so does Python, but timing how things are dispatched carries almost no information about how fast Python is. It’s missleading information to someone who is trying to learn new language for example.

[Oscar_Smith]

I disagree with this. It’s not unreasonable to expect Julia to be faster than the C code python calls, and when bench-marking sorting vs python, the relevant time is the timsort vs the default Julia sort. This isn’t a great sorting benchmark for other reasons (i.e. most of the time is IO), but it is a decent benchmark of doing basic data science in Julia vs python.

[Mateusz_K]

To me Julia’s website has already benchmarks done properly, comparing what it claims to be comparing. There C seems to be baseline for most of tests. Why would it be slower ? Just have a look. You typically don’t benchmark numpy calls to tell that python is fast, because the very reason of having numpy in the first place is that Python is absolutely slow. But if you do make such a claim, be honest and say it’s numpy’s speed. Regarding data science, lots of people mean different thing by that, but for applications where sort() call speed matters, is probably not the application you benefit from Python. I use Python a lot but it’s absolutely a terrible tool for applications where speed matters, such as algorithms development. As soon as you want something that is not in toolbox you are screwed. Most of stuff your run in Python is not even Python because authors that prise it so much shy from implementing it in their very own favourite language.

[adienes]

In the end, everything just calls machine instructions. I think it’s very reasonable to measure the “speed” of a language based on how easy it is to write performant code. For many use cases, Python is fast because numpy is fast. I don’t think that’s some kind of “gotcha,” it’s just true.

it’s absolutely a terrible tool for applications where speed matters

well, except for nearly 100% of mainstream deep learning