Code written in Julia vs code written using PyCall

New to Julia
1

It’s claimed that Julia is faster than Python. Is it still true in a case if I use code from Python using PyCall?

2

Yes, PyCall is still using Python to run the code and does not transform Python code to julia code

1 Like
3

So no, it is not true that Julia is still faster than Python in those cases.

1 Like
4

Yeah, I misread the question as “Can I speed python up by calling it from julia” rather than “Is julia still faster than python, if I use PyCall”.

1 Like
5

You should of course bear in mind that there is an enormous amount of subtlety involved in comparing the “speed” of languages. In particular, the reason Python works as well as it does is that the most important Python packages (e.g. numpy, scipy) are not written in Python they are written in C and FORTRAN. These packages. as well as some of the core Python data structures such as strings and dictionaries are very mature and highly optimized. Many of the simplest things you can do in Python will be nearly as fast as they are in Julia (for example, a dict lookup). The thing that goes horribly wrong in Python is that any time you want to do something with your actual Python code that is not just calling some black-box algorithm from C, it grinds to a snail’s pace. Of course, this problem is a huge part of the motivation for Julia so it is discussed extensively in these forums and elsewhere.

PyCall essentially just calls Python code using the Python interpreter to provide you with Python compatibility in Julia. As @BeastyBlacksmith pointed out, it does not transform any of the Python code to Julia code, so anything called through PyCall is running entirely in Python and then getting loaded into Julia. There is of course some additional overhead in this import process, which, as far as I can tell, is mostly just due diligence to make sure the result of Python calls is something sane, no errors were thrown, etc. In some cases binary formats are completely different and they need to be reformated (fortunately in one of the most important cases, numpy arrays of bits types, no reformating is necessary).

So, in summary, Python code called through PyCall is a little bit “slower” than Python code called from Python.

4 Likes
6

It’s more like a lot slower:

julia> using PyCall

julia> function fib(f, n)
           if n < 2
               return n
           end
           return f(fib, n - 2) + f(fib, n - 1)
       end
fib (generic function with 1 method)

julia> py"""
       def fib(f, n):
           if n < 2:
               return n
           return f(fib, n - 2) + f(fib, n - 1)
       """

julia> using BenchmarkTools

julia> @benchmark fib($(py"fib"), 15)  # Python-Julia hybrid
BenchmarkTools.Trial:
  memory estimate:  955.20 KiB
  allocs estimate:  36483
  --------------
  minimum time:     10.525 ms (0.00% GC)
  median time:      11.148 ms (0.00% GC)
  mean time:        12.164 ms (5.64% GC)
  maximum time:     79.173 ms (71.71% GC)
  --------------
  samples:          411
  evals/sample:     1

julia> @benchmark $(py"lambda n: fib(fib, n)")(15)  # pure Python
BenchmarkTools.Trial:
  memory estimate:  48 bytes
  allocs estimate:  3
  --------------
  minimum time:     135.319 μs (0.00% GC)
  median time:      136.029 μs (0.00% GC)
  mean time:        139.117 μs (0.00% GC)
  maximum time:     349.509 μs (0.00% GC)
  --------------
  samples:          10000
  evals/sample:     1

julia> @benchmark fib($fib, 15)  # pure Julia
BenchmarkTools.Trial:
  memory estimate:  16 bytes
  allocs estimate:  1
  --------------
  minimum time:     2.508 μs (0.00% GC)
  median time:      2.515 μs (0.00% GC)
  mean time:        2.667 μs (0.00% GC)
  maximum time:     7.248 μs (0.00% GC)
  --------------
  samples:          10000
  evals/sample:     9

(though you can squeeze out performance a little bit with pyfunctionret)

But this is of course not the intended usecase especially when you need the speed.

3 Likes