# Switched from Python to Julia for speed, but it's not faster...?

**URL:** <https://discourse.julialang.org/t/switched-from-python-to-julia-for-speed-but-its-not-faster/45904>\
**Category:** Performance\
**Tags:** python\
**Created:** [September 1, 2020, 3:44pm UTC](https://discourse.julialang.org/t/switched-from-python-to-julia-for-speed-but-its-not-faster/45904 "2020-09-01T15:44:22Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![PaulScotti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/paulscotti/32/17623_2.png) [@PaulScotti](https://discourse.julialang.org/u/PaulScotti)\
**Post date:** [September 1, 2020, 3:44pm UTC](https://discourse.julialang.org/t/switched-from-python-to-julia-for-speed-but-its-not-faster/45904/1 "2020-09-01T15:44:22Z")

</div>

I have a Python script where the major speed hurdle is a single function in the statsmodels package. However, it seems that if I use PyCall to use the same Python function in Julia, the speed hurdle persists to the same degree. Is it true that I need to rewrite the function (i.e., no pyimport) in native Julia for me to expect a speed boost? I was maybe under the mistaken assumption that simply porting over my Python code to Julia would make everything faster…

Here are two identical scripts in Python and Julia, with the corresponding speeds to complete:

Python (64.7 seconds):

```julia
import statsmodels.tsa.stattools as statts
cur_time=datetime.datetime.now()
T1=ts[0,:]
T2=ts[1,:]
coint = statts.coint(T1,T2,trend='ct')[1]
print(datetime.datetime.now()-cur_time)

```

Julia (68.155 seconds):

```julia
statts = pyimport("statsmodels.tsa.stattools")
cur_time=datetime.datetime.now()
T1=ts[1,:]
T2=ts[2,:]
coint = statts.coint(T1,T2,trend="ct")[2]
print(datetime.datetime.now()-cur_time)

```

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [September 1, 2020, 3:50pm UTC](https://discourse.julialang.org/t/switched-from-python-to-julia-for-speed-but-its-not-faster/45904/2 "2020-09-01T15:50:40Z")

</div>

> [@PaulScotti](#):
>
> I was maybe under the mistaken assumption that simply porting over my Python code to Julia would make everything faster…

PyCall just calls into Python so it will execute exactly the same as if you ran it in Python.

---

<div class="post-metadata">

**Author:** ![tbeason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tbeason/32/15898_2.png) [@tbeason](https://discourse.julialang.org/u/tbeason)\
**Post date:** [September 1, 2020, 4:04pm UTC](https://discourse.julialang.org/t/switched-from-python-to-julia-for-speed-but-its-not-faster/45904/3 "2020-09-01T16:04:38Z")

</div>

> [@PaulScotti](#):
>
> I was maybe under the mistaken assumption that simply porting over my Python code to Julia would make everything faster…

That’s not a mistaken assumption, it’s correct. That’s just not what you did. “Porting” code generally means rewriting it in a new language.

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [September 1, 2020, 4:07pm UTC](https://discourse.julialang.org/t/switched-from-python-to-julia-for-speed-but-its-not-faster/45904/4 "2020-09-01T16:07:37Z")

</div>

Also very likely that function is implemented in C or Fortran under the hood in the python implementation, and thus transporting a single call of that function from python to julia, or rewriting the function in julia won’t bring any important speedup. The advantage of Julia is that IF you rewrite that function in pure Julia you may get similar speeds than the C our Fortran code python is calling, without having to deal with two different languages and their interfaces.

---

<div class="post-metadata">

**Author:** ![StefanKarpinski](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefankarpinski/32/24_2.png) [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)\
**Post date:** [September 1, 2020, 4:13pm UTC](https://discourse.julialang.org/t/switched-from-python-to-julia-for-speed-but-its-not-faster/45904/5 "2020-09-01T16:13:05Z")

</div>

Related:

[Frequently Asked Questions · The Julia Language](https://docs.julialang.org/en/v1/manual/faq/#Why-don't-you-compile-Matlab/Python/R/%E2%80%A6-code-to-Julia)?

If calling Python code as-is from Julia could somehow make it magically faster without changing its behavior, then Python itself would just do that and already be faster. The reason that Julia is faster than Python is that it works differently.

---

<div class="post-metadata">

**Author:** ![xiaodai](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xiaodai/32/15937_2.png) [@xiaodai](https://discourse.julialang.org/u/xiaodai)\
**Post date:** [September 1, 2020, 4:56pm UTC](https://discourse.julialang.org/t/switched-from-python-to-julia-for-speed-but-its-not-faster/45904/6 "2020-09-01T16:56:48Z")

</div>

> [@PaulScotti](#):
>
> I was maybe under the mistaken assumption that simply porting over my Python code to Julia would make everything faster…

Maybe something like PyPy that doesn’t involve the C interface would be magically faster. But that’s faster interpreter due to JITting. And Julia is not a python interpreter. I wonder how prevalent this misconception is?

This is the first time I have come across this particular misconception.

---

<div class="post-metadata">

**Author:** ![PaulScotti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/paulscotti/32/17623_2.png) [@PaulScotti](https://discourse.julialang.org/u/PaulScotti)\
**Post date:** [September 1, 2020, 5:39pm UTC](https://discourse.julialang.org/t/switched-from-python-to-julia-for-speed-but-its-not-faster/45904/7 "2020-09-01T17:39:48Z")

</div>

Yeah, I guess my misconception does come off as stupid. In my mind, the syntax between Julia/Matlab/Python are very similar – I thought for most Python packages there was a way to automatically translate between the languages on the back-end, such that PyCall is doing the work of converting Python functions into native Julia. I was watching YouTube videos about how Julia is so much faster than Python and it can even already handle all my Python packages – this is not as amazing sounding if I knew that if I did use my Python packages then they don’t get the Julia benefits the speaker was talking about.

Regardless, thanks for the replies! I’ll have to spend more time with this language to better undestand things.

---

<div class="post-metadata">

**Author:** ![mbauman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mbauman/32/31082_2.png) [@mbauman](https://discourse.julialang.org/u/mbauman)\
**Post date:** [September 1, 2020, 5:47pm UTC](https://discourse.julialang.org/t/switched-from-python-to-julia-for-speed-but-its-not-faster/45904/8 "2020-09-01T17:47:16Z")

</div>

This isn’t uncommon at all — and you shouldn’t feel stupid for asking it. It can be tricky to simultaneously talk about how great Julia itself is _and_ how great Julia’s interoperability is without some muddy waters.

The beautiful thing about the interoperability is that you don’t need to replace your entire codebase — you can incrementally update the parts that are taking the biggest amounts of time. The tricky part is that Python isn’t written in Python (largely), so if your Python usage is largely calling into highly-optimized libraries (written in C/C++/Fortran) you may not see a significant speedup even after converting to Julia. Where you’ll see the biggest speed gains are when you’re writing your own `for` loops and algorithms — or possibly calling some not-so-optimal libraries.

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [September 1, 2020, 5:48pm UTC](https://discourse.julialang.org/t/switched-from-python-to-julia-for-speed-but-its-not-faster/45904/9 "2020-09-01T17:48:44Z")

</div>

Effectively, if the speed hurdle is on the functions you call from your python scripts, and those functions are implemented in low-level languages, you won’t get a great benefit from using Julia instead. If, on the other side, the speed hurdle is in your python script, or in functions written in python itself, to solve those hurdles you will probably have to re-implement those functions in a low level language or, now, in Julia.

---

<div class="post-metadata">

**Author:** ![Jordan\_Cluts](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jordan_cluts/32/13753_2.png) [@Jordan\_Cluts](https://discourse.julialang.org/u/Jordan_Cluts)\
**Post date:** [September 1, 2020, 6:21pm UTC](https://discourse.julialang.org/t/switched-from-python-to-julia-for-speed-but-its-not-faster/45904/10 "2020-09-01T18:21:47Z")

</div>

> [@xiaodai](#):
>
> This is the first time I have come across this particular misconception.

I have run into multiple people in real life in my industry (scientific research/engineering) who think that Julia is a new Python interpreter implementation.

---

<div class="post-metadata">

**Author:** ![xiaodai](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xiaodai/32/15937_2.png) [@xiaodai](https://discourse.julialang.org/u/xiaodai)\
**Post date:** [September 1, 2020, 11:36pm UTC](https://discourse.julialang.org/t/switched-from-python-to-julia-for-speed-but-its-not-faster/45904/11 "2020-09-01T23:36:17Z")

</div>

Yeah I can sympathise.

I used to think renaming the extension would compilation…

---

<div class="post-metadata">

**Author:** ![dlakelan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlakelan/32/8491_2.png) [@dlakelan](https://discourse.julialang.org/u/dlakelan)\
**Post date:** [September 2, 2020, 2:13am UTC](https://discourse.julialang.org/t/switched-from-python-to-julia-for-speed-but-its-not-faster/45904/12 "2020-09-02T02:13:01Z")

</div>

> [@lmiq](#):
>
> Effectively, if the speed hurdle is on the functions you call from your python scripts, and those functions are implemented in low-level languages, you won’t get a great benefit from using Julia instead

I will say that Julia offers you a lot of great tools to write good code. If you decide to implement some functionality in Julia you may find that it is faster because you implement a more sophisticated algorithm for solving that problem since Julia makes working with more sophisticated ideas simpler.

For example, while Python has dictionaries, C doesn’t. So if python is calling C to do the “fast stuff” but the C programmers are using some algorithm that doesn’t take into account the properties of a dictionary, then reimplementing the thing in Julia and taking into account that there are dictionaries could give you a massive improvement.

Similarly for many things in DataStructures.jl like Deques or Priority Queues or Disjoint Sets or whatever.

most of the time the big improvements come from better algorithms. Julia makes writing sophisticated algorithms a LOT nicer than doing so in C or C++ IMHO

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [September 2, 2020, 2:51am UTC](https://discourse.julialang.org/t/switched-from-python-to-julia-for-speed-but-its-not-faster/45904/13 "2020-09-02T02:51:43Z")

</div>

I completely agree. My point was that for someone only using python to call internal libraries to do most of the work, the benefits of Julia might not be clear. This is absolutely not my case, for example. Being used to program in Fortran, I always disliked python, because I had to adapt my problems to the use of those libraries, when they existed, and if they didn’t exist I had to go back to Fortran. With Julia I can both use powerful libraries if I want and they are available, or write my own code. But I understand that many people are just using the language as a parser for black-box algorithms and, in that case, not much can be improved by changing the language.