# General questions from Python user

**URL:** <https://discourse.julialang.org/t/general-questions-from-python-user/55475>\
**Category:** Performance\
**Created:** [February 17, 2021, 5:23pm UTC](https://discourse.julialang.org/t/general-questions-from-python-user/55475 "2021-02-17T17:23:32Z")\
**Posts on this page:** 20\
**Page:** 2

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [February 22, 2021, 5:39pm UTC](https://discourse.julialang.org/t/general-questions-from-python-user/55475/21 "2021-02-22T17:39:06Z")

</div>

> [@jling](#):
>
> in global scope sweet heart:

on purpose

---

<div class="post-metadata">

**Author:** ![jling](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jling/32/212909_2.png) [@jling](https://discourse.julialang.org/u/jling)\
**Post date:** [February 22, 2021, 5:39pm UTC](https://discourse.julialang.org/t/general-questions-from-python-user/55475/22 "2021-02-22T17:39:50Z")

</div>

🤷‍♂️ assuming people purposefully use global scope to perform unrealistic task just to show Julia is slow, I guess this demo is applicable.

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [February 22, 2021, 5:43pm UTC](https://discourse.julialang.org/t/general-questions-from-python-user/55475/23 "2021-02-22T17:43:27Z")

</div>

I just meant to show that if one writes type-unstable code (in this case, on purpose), performance is bad, and probably that is what “naive” may mostly mean in Julia.

---

<div class="post-metadata">

**Author:** ![lungben](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lungben/32/12314_2.png) [@lungben](https://discourse.julialang.org/u/lungben)\
**Post date:** [February 22, 2021, 5:49pm UTC](https://discourse.julialang.org/t/general-questions-from-python-user/55475/24 "2021-02-22T17:49:02Z")

</div>

By replacing `zeros` with `similar` (which just allocates the output array without setting it to 0) and adding `@inbounds` the function gives the same performance as broadcasting:

```julia
function f2(a,b)
    y = similar(a)
    @inbounds for i in 1:length(a)
        y[i] = a[i]/b[i]
    end
    y
end

```

But I agree that this is no longer naive 😉  
Coming from Python / Numpy, broadcasting seemed to be the most natural way to do “vectorized” operations in Julia.

---

<div class="post-metadata">

**Author:** ![jling](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jling/32/212909_2.png) [@jling](https://discourse.julialang.org/u/jling)\
**Post date:** [February 22, 2021, 5:50pm UTC](https://discourse.julialang.org/t/general-questions-from-python-user/55475/25 "2021-02-22T17:50:37Z")

</div>

but even in this case julia is still faster:

```julia
In [7]: %%timeit
   ...: for i in range(0,len(y)):
   ...: y[i] = a[i]/b[i]
   ...:
   ...:
229 µs ± 2.87 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

julia> @btime for i in 1:length(a)
         y[i] = a[i] / b[i]
       end
  97.144 μs (4980 allocations: 93.45 KiB)

```

you can’t assume user knows to vectorize in numpy but don’t know broadcasting in Julia.

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [February 22, 2021, 5:51pm UTC](https://discourse.julialang.org/t/general-questions-from-python-user/55475/26 "2021-02-22T17:51:13Z")

</div>

> [@lungben](#):
>
> But I agree that this is no longer naive

Exactly. This is correct. It is easy to write fast Julia code, if one is aware of the basic stuff. But there is an initial set of things to learn and get used to (as expected for everything that does not do magic).

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [February 22, 2021, 5:54pm UTC](https://discourse.julialang.org/t/general-questions-from-python-user/55475/27 "2021-02-22T17:54:09Z")

</div>

> [@jling](#):
>
> you can’t assume user knows to vectorize in numpy but don’t know broadcasting in Julia.

I think we do not disagree in anything.

---

<div class="post-metadata">

**Author:** ![Skoffer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/skoffer/32/378_2.png) [@Skoffer](https://discourse.julialang.org/u/Skoffer)\
**Post date:** [February 22, 2021, 6:04pm UTC](https://discourse.julialang.org/t/general-questions-from-python-user/55475/28 "2021-02-22T18:04:07Z")

</div>

Naive implementation usually also includes slicing without views. Since numpy slice are equivalent to Julia views, direct translation of numpy code can result in huge allocations and as a result very bad performance.

---

<div class="post-metadata">

**Author:** ![apo383](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/apo383/32/11272_2.png) [@apo383](https://discourse.julialang.org/u/apo383)\
**Post date:** [February 22, 2021, 6:04pm UTC](https://discourse.julialang.org/t/general-questions-from-python-user/55475/29 "2021-02-22T18:04:21Z")

</div>

That’s of course correct, but it seems like this case is the most common demonstration of “Julia is slower than Python.” People tend to:

- benchmark in global scope
- without running once to compile, and
- without interpolating their globals with `$`
- and often with a type instability

It’s understandable why they do this. They see a neat blog with a quick Jupyter notebook, they download Julia, and want to try a couple things out. It’s probably necessary to read several different parts of the manual to get a proper, non-naive benchmark.

---

<div class="post-metadata">

**Author:** ![Eben60](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/eben60/32/13475_2.png) [@Eben60](https://discourse.julialang.org/u/Eben60)\
**Post date:** [February 22, 2021, 7:42pm UTC](https://discourse.julialang.org/t/general-questions-from-python-user/55475/30 "2021-02-22T19:42:22Z")

</div>

> [@jling](#):
>
> Unless by naive you mean intentionally type unstable. Example please?

_Naive_ is not a synonyme for _plain_, nor for _evil_. Oxford Languages defines it as “showing a lack of experience, wisdom, or judgement”, or “natural and unaffected; innocent”. I would thus define it as having the best intentions, but little expertise.

Some Julia-specific ways of _ **un** _intentionally shooting oneself in the foot were listed in the comments above. It is easy to get a slow-down by two orders of magnitude. An example of a slow-down by a factor of 70 due to unnecessary allocations was actually cited in my previous post.

---

<div class="post-metadata">

**Author:** ![jling](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jling/32/212909_2.png) [@jling](https://discourse.julialang.org/u/jling)\
**Post date:** [February 22, 2021, 8:46pm UTC](https://discourse.julialang.org/t/general-questions-from-python-user/55475/31 "2021-02-22T20:46:09Z")

</div>

> [@Eben60](#):
>
> An example of a slow-down by a factor of 70

respect to a more optimal Julia code, sure. But what about when compared to Python / Numpy? Your last statement was:

> [@Eben60](#):
>
> **much** slower (e.g. naive Julia against numpy)

but that [post](https://discourse.julialang.org/t/problem-with-huge-number-of-memory-allocations-in-for-loops/55714) was about C++. Now, I think everyone can agree there are more quicks in C++ than Python or Julia.

---

<div class="post-metadata">

**Author:** ![Eben60](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/eben60/32/13475_2.png) [@Eben60](https://discourse.julialang.org/u/Eben60)\
**Post date:** [February 22, 2021, 9:15pm UTC](https://discourse.julialang.org/t/general-questions-from-python-user/55475/32 "2021-02-22T21:15:00Z")

</div>

Now I could probably write a sufficiently fast solution of the cited problem in Python. I wouldn’t. You won, Julia is the best, and inherently better than Python.

For everyone else - see above. Thank you.

---

<div class="post-metadata">

**Author:** ![jling](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jling/32/212909_2.png) [@jling](https://discourse.julialang.org/u/jling)\
**Post date:** [February 22, 2021, 9:17pm UTC](https://discourse.julialang.org/t/general-questions-from-python-user/55475/33 "2021-02-22T21:17:51Z")

</div>

> [@Eben60](#):
>
> Julia is the best, and inherently better than Python.

straw man fallacy is not fun. That was not my argument at all.

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [February 22, 2021, 11:26pm UTC](https://discourse.julialang.org/t/general-questions-from-python-user/55475/34 "2021-02-22T23:26:32Z")

</div>

There have been heaps of threads like “Why is my code translated from Python so much slower in Julia?” I don’t understand the point of denying that naive Julia code can be slower than Python.

---

<div class="post-metadata">

**Author:** ![jling](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jling/32/212909_2.png) [@jling](https://discourse.julialang.org/u/jling)\
**Post date:** [February 22, 2021, 11:45pm UTC](https://discourse.julialang.org/t/general-questions-from-python-user/55475/35 "2021-02-22T23:45:21Z")

</div>

I’m yet to see a computationally heavy task (i.e. not about julia starting time is slow) that is written in either: both using for loop, or, both in vectorized/broadcasting style; that shows Julia to be much slower than Python/Numpy.

---

<div class="post-metadata">

**Author:** ![Mason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mason/32/2423_2.png) [@Mason](https://discourse.julialang.org/u/Mason)\
**Post date:** [February 22, 2021, 11:52pm UTC](https://discourse.julialang.org/t/general-questions-from-python-user/55475/36 "2021-02-22T23:52:40Z")

</div>

The point is just that if one doesn’t bother to learn Julian idioms and techniques, and instead just ‘writes Python’ in Julia with surface level syntax changes, it’s absolutely quite easy to end up with massive performance problems.

Perhaps the best example would just be tight loops involving global variables, or creating empty, I typed arrays and then pushing to them. Sure, it’s pretty easy to learn how to avoid these problems, but that’s not the point.

---

<div class="post-metadata">

**Author:** ![jling](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jling/32/212909_2.png) [@jling](https://discourse.julialang.org/u/jling)\
**Post date:** [February 23, 2021, 12:12am UTC](https://discourse.julialang.org/t/general-questions-from-python-user/55475/37 "2021-02-23T00:12:25Z")

</div>

for the record, I just want to amend to:

> [@Mason](#):
>
> tight loops involving global variables, or creating empty, I typed arrays and then pushing to them

that, in these practices Python will be slow too (due to similar reason, being a dynamic language itself) and very often slower than Julia:

```julia
julia> a = rand(10^5);

julia> b = 0;

julia> @btime for x in a
           if x > 0.5
               global b+=x
           end
       end
  6.738 ms (349237 allocations: 6.85 MiB)

In [16]: a = np.random.rand(10**5)

In [17]: b=0

In [18]: %%timeit
    ...: for x in a:
    ...: global b
    ...: if x > 0.5:
    ...: b+=x
    ...:
13 ms ± 325 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

```

But I think everyone in the thread have something to takeaway and I guess I should stop being annoying.

---

<div class="post-metadata">

**Author:** ![Mason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mason/32/2423_2.png) [@Mason](https://discourse.julialang.org/u/Mason)\
**Post date:** [February 23, 2021, 12:29am UTC](https://discourse.julialang.org/t/general-questions-from-python-user/55475/38 "2021-02-23T00:29:00Z")

</div>

> [@jling](#):
>
> that, in these practices Python will be slow too (due to similar reason, being a dynamic language itself) and very often slower than Julia:

It can sometimes carry a higher cost in julia however.

If you’d like an example, here’s an example from [this thread](https://discourse.julialang.org/t/why-is-python-faster-than-julia/35890/5):

```julia
In [6]: def euclidian_algorithm_division_count(a, b):
   ...: division_count = 1
   ...: if b > a:
   ...: a, b = b, a
   ...: while (c := a % b) != 0:
   ...: a, b = b, c
   ...: division_count += 1
   ...: return division_count
   ...: 
   ...: from random import randint

In [7]: %%timeit
   ...: N = 10**100
   ...: M = 10**4
   ...: division_count_array = []
   ...: while M > 0:
   ...: a = randint(1, N)
   ...: b = randint(1, N)
   ...: division_count_array.append(euclidian_algorithm_division_count(a, b))
   ...: M -= 1
292 ms ± 7.74 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

```

```julia
julia> function euclidean_algorithm_division_count(a, b)
           division_count = 1
           if b > a
               a, b = b, a
           end
           while (c = a % b) != 0
               a, b = b, c
               division_count += 1
           end
           return division_count
       end
euclidean_algorithm_division_count (generic function with 1 method)

julia> function main()
           N = big(10)^100
           M = 10^4
           division_count_array = []
           while M > 0
               a, b = rand(1:N, 2)
               push!(division_count_array, euclidean_algorithm_division_count(a, b))
               M -= 1
           end
       end
main (generic function with 1 method)

julia> @btime main()
  378.040 ms (5618922 allocations: 110.55 MiB)

```

It’s of course not very hard to make the julia version beat the Python version, but this straightforward transcription (that even uses a function) of naive Python code can still be slower in julia.

---

<div class="post-metadata">

**Author:** ![jling](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jling/32/212909_2.png) [@jling](https://discourse.julialang.org/u/jling)\
**Post date:** [February 23, 2021, 12:32am UTC](https://discourse.julialang.org/t/general-questions-from-python-user/55475/39 "2021-02-23T00:32:50Z")

</div>

Thanks for bearing with me! This is a pretty neat pedagogical example!

Ah, looks like a BigInt issue, less interesting than I initially thought.

---

<div class="post-metadata">

**Author:** ![moeddel](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/moeddel/32/18641_2.png) [@moeddel](https://discourse.julialang.org/u/moeddel)\
**Post date:** [February 23, 2021, 1:00pm UTC](https://discourse.julialang.org/t/general-questions-from-python-user/55475/40 "2021-02-23T13:00:30Z")

</div>

> [@jling](#):
>
> that, in these practices Python will be slow too (due to similar reason, being a dynamic language itself) and very often slower than Julia:

be careful with your timings though. `timeit` does report the mean, whereas `btime` reports the minimum run time.

[Previous page](https://discourse.julialang.org/t/general-questions-from-python-user/55475.md?page=1)

[Next page](https://discourse.julialang.org/t/general-questions-from-python-user/55475.md?page=3)