# Julia is slower than Python when appending elements to untyped arrays

**URL:** <https://discourse.julialang.org/t/julia-is-slower-than-python-when-appending-elements-to-untyped-arrays/125745>\
**Category:** General Usage\
**Tags:** question\
**Created:** [February 10, 2025, 1:09pm UTC](https://discourse.julialang.org/t/julia-is-slower-than-python-when-appending-elements-to-untyped-arrays/125745 "2025-02-10T13:09:00Z")\
**Posts on this page:** 20\
**Page:** 2

<div class="post-metadata">

**Author:** ![karei](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/karei/32/214809_2.png) [@karei](https://discourse.julialang.org/u/karei)\
**Post date:** [February 10, 2025, 3:13pm UTC](https://discourse.julialang.org/t/julia-is-slower-than-python-when-appending-elements-to-untyped-arrays/125745/21 "2025-02-10T15:13:58Z")

</div>

If you write type-unstable code, then as jakobnissen says, “you’re right that when the Julia compiler has _no_ information about types, Julia is typically slower than Python. After all, Python’s interpreter and the entire language has been optimised for exactly that scenario.”  
When you write type-stable code (which may take you more time), Julia can approach C speed. That’s all.

---

<div class="post-metadata">

**Author:** ![JM\_Beckers](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jm_beckers/32/22482_2.png) [@JM\_Beckers](https://discourse.julialang.org/u/JM_Beckers)\
**Post date:** [February 10, 2025, 3:15pm UTC](https://discourse.julialang.org/t/julia-is-slower-than-python-when-appending-elements-to-untyped-arrays/125745/22 "2025-02-10T15:15:44Z")

</div>

> [@world-peace](#):
>
> An equivalent Julia code to this small Python benchmark runs nearly 10x slower

In addition of questioning the relevance of the benchmark, I notice that on my computer your codes run at almost the same speed (Julia being even a little bit faster).

---

<div class="post-metadata">

**Author:** ![world-peace](https://avatars.discourse-cdn.com/v4/letter/w/9f8e36/32.png) [@world-peace](https://discourse.julialang.org/u/world-peace)\
**Post date:** [February 10, 2025, 3:27pm UTC](https://discourse.julialang.org/t/julia-is-slower-than-python-when-appending-elements-to-untyped-arrays/125745/23 "2025-02-10T15:27:43Z")

</div>

> [@sdanisch](#):
>
> Why not?  
> Julia _isn’t_ used that way for high performance code, so why should it be heavily optimized?

Because what the compiler should be doing is very, very simple. And the code is entirely _compiled_ there is none of the additional overhead which Python suffers from.

---

<div class="post-metadata">

**Author:** ![Benny](https://avatars.discourse-cdn.com/v4/letter/b/49beb7/32.png) [@Benny](https://discourse.julialang.org/u/Benny)\
**Post date:** [February 10, 2025, 3:34pm UTC](https://discourse.julialang.org/t/julia-is-slower-than-python-when-appending-elements-to-untyped-arrays/125745/24 "2025-02-10T15:34:32Z")

</div>

For those who are actually curious about what CPython’s built-in `sum` does, here’s [a link](https://github.com/python/cpython/blob/d7672e5d5a7b9580a72dbe75d3a9e8840bcc604c/Python/bltinmodule.c#L2682), and it’s not hard to find a reason why it does well for an iterable of `int`. There is no truth to the implication that CPython is inherently slow because it’s “not compiled”; there is actually a decent chunk of compiled code in CPython, the language hinted in the name. There’s nothing preventing CPython from implementing Julia’s much more efficient `sum(::AbstractRange)` for its built-in `range`, either.

> [@world-peace](#):
>
> And the code is entirely _compiled_ there is none of the additional overhead

This is false, and people have told you how in many threads. Compilers are not magic overhead disappearers.

---

<div class="post-metadata">

**Author:** ![sdanisch](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sdanisch/32/1406_2.png) [@sdanisch](https://discourse.julialang.org/u/sdanisch)\
**Post date:** [February 10, 2025, 3:39pm UTC](https://discourse.julialang.org/t/julia-is-slower-than-python-when-appending-elements-to-untyped-arrays/125745/25 "2025-02-10T15:39:43Z")

</div>

This shouldn’t be so hard to understand…

If you take a language, who’s whole point is to be faster than python **because** it has more type information, and then you take away the type information, it should be pretty obvious that the result won’t necessarily be a win.

Of course, Julia _could_ have the same speed here, if it had 20 years of optimizing untyped code under it’s belly - which it simply hasn’t since it isn’t a priority.  
I’m sure you’re well aware how difficult it is, to improve pythons performance, and a TON of money has been thrown onto the problem of e.g. optimizing untyped arrays. If this was a simple problem, Python would easily match Julia speed for highly typed Julia code and Julia / C wouldn’t be required at all.

It’s really like throwing a Cheetah into water and complaining that it isn’t swimming as fast as a fish.

---

<div class="post-metadata">

**Author:** ![world-peace](https://avatars.discourse-cdn.com/v4/letter/w/9f8e36/32.png) [@world-peace](https://discourse.julialang.org/u/world-peace)\
**Post date:** [February 10, 2025, 3:47pm UTC](https://discourse.julialang.org/t/julia-is-slower-than-python-when-appending-elements-to-untyped-arrays/125745/26 "2025-02-10T15:47:15Z")

</div>

None of you are compiler engineers, so I don’t think your opinions count for much to be honest. None of you have seen an x86 instruction before.

As for posting a link to the CPython code and saying “see that’s why it’s fast”.

No, I don’t see why it is fast. Look at all the overhead with the many function calls there to sanity check various conditions during the iteration.

Looking at this code just makes me think all the more that the Julia compiler **must** be producing poor performing or inefficient object code.

Put it this way, for the non-experts in this thread.

- Python follows a reference for each element of the list, then does some runtime lookup to see how to add the type of the current value to the type of the current sum value
- This is very similar to what Julia does, except that Python has some overhead due to the fact the function call starts off as Python code before calling into a C level implementation. Since Julia has all the “magic” as others have put it (it isn’t magic) of being JIT-d you would **expect** if efficient machine code was being produced and thus there was none of the overhead which Python suffers from, it **should** be faster but it is not

---

<div class="post-metadata">

**Author:** ![sdanisch](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sdanisch/32/1406_2.png) [@sdanisch](https://discourse.julialang.org/u/sdanisch)\
**Post date:** [February 10, 2025, 3:48pm UTC](https://discourse.julialang.org/t/julia-is-slower-than-python-when-appending-elements-to-untyped-arrays/125745/27 "2025-02-10T15:48:39Z")

</div>

> None of you are compiler engineers, so I don’t think your opinions count for much to be honest. None of you have seen an x86 instruction before.

Haha 😃 Ok, why the hell would you even assume this?

Just to prove my point, here’s a very simple version of how an optimized version could look like:

```julia

struct UntypedArray 
    data::Dict{DataType, Vector}
end

UntypedArray() = UntypedArray(Dict{DataType, Vector}())
function Base.push!(data::UntypedArray, value::T) where T
    if !haskey(data.data, T)
        data.data[T] = Vector{T}()
    end
    push!(data.data[T], value)
end

function Base.sum(data::UntypedArray)
    if length(data.data) == 1
        sum(last(first(data.data)))
    else
        throw(ArgumentError("Cannot sum an array with more than one type"))
    end
end

function main()

    N = 100000000
    data = UntypedArray()

    for i in 1:N
        push!(data, i)
    end

    # throw 1 call away, JIT
    total = sum(data)

    println("starting")
    time_start = Base.time_ns()
    total = sum(data)
    time_stop = Base.time_ns()
    println("finished")
    duration = time_stop - time_start
    duration = 1.0e-3 * duration
    println("time taken: $(duration) us")
    println("total: $(total)")
end

```

This is quite a bit faster than the Python version.

Obviously, it’s an incomplete implementation, but with a bit more time you could implement something like this in Julia pretty efficiently which would match pythons speed and would be a complete array implementation.

I don’t know the details of the python sum and array implementation, but I’m pretty sure they have tried hard to come up with something that works really well for untyped elements.

---

<div class="post-metadata">

**Author:** ![world-peace](https://avatars.discourse-cdn.com/v4/letter/w/9f8e36/32.png) [@world-peace](https://discourse.julialang.org/u/world-peace)\
**Post date:** [February 10, 2025, 3:49pm UTC](https://discourse.julialang.org/t/julia-is-slower-than-python-when-appending-elements-to-untyped-arrays/125745/28 "2025-02-10T15:49:50Z")

</div>

As for claiming nonsense about “not providing type information so it’s slow” this doesn’t even make sense for a comparison.

We cannot compare applies to oranges here.

If you want a strong type comparison, compare Julia to Python + Numpy or Julia to C/C++. But this **has nothing to do with what I am asking about** , as I already said I am working in an environment where we have objects of different types in a list.

---

<div class="post-metadata">

**Author:** ![world-peace](https://avatars.discourse-cdn.com/v4/letter/w/9f8e36/32.png) [@world-peace](https://discourse.julialang.org/u/world-peace)\
**Post date:** [February 10, 2025, 3:52pm UTC](https://discourse.julialang.org/t/julia-is-slower-than-python-when-appending-elements-to-untyped-arrays/125745/29 "2025-02-10T15:52:06Z")

</div>

Why de-rail the thread by posting this irrelevant information? This is not “more optimized” you just chucked a bunch of different type vectors into a dict.

What is anyone supposed to conclude from you solving a different problem to the original one posed?

---

<div class="post-metadata">

**Author:** ![mbauman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mbauman/32/31082_2.png) [@mbauman](https://discourse.julialang.org/u/mbauman)\
**Post date:** [February 10, 2025, 3:53pm UTC](https://discourse.julialang.org/t/julia-is-slower-than-python-when-appending-elements-to-untyped-arrays/125745/30 "2025-02-10T15:53:49Z")

</div>

@world-peace, what rails would you _like_ this thread to ride on? What’s your goal here?

---

<div class="post-metadata">

**Author:** ![sdanisch](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sdanisch/32/1406_2.png) [@sdanisch](https://discourse.julialang.org/u/sdanisch)\
**Post date:** [February 10, 2025, 3:56pm UTC](https://discourse.julialang.org/t/julia-is-slower-than-python-when-appending-elements-to-untyped-arrays/125745/31 "2025-02-10T15:56:04Z")

</div>

My point is, that because the Base array implementation isn’t optimized for `sum(x::Vector{Any})`, one would need a new implementation that has been optimized for this use case.  
The Julia one **isn’t** optimized for this use case, which is why its slow. I’m not sure why it’s such a hard pill to swallow.

The implementation is shitty and pointless, but I won’t create a decent implementation of an array type that’s better at iterating over any elements just to prove a point for an internet argument.

If you haven’t noticed so far, it’s actually pretty normal in programming, to optimize a data structure for one use case, and take penalties for other use cases which occur less often 😉

---

<div class="post-metadata">

**Author:** ![Benny](https://avatars.discourse-cdn.com/v4/letter/b/49beb7/32.png) [@Benny](https://discourse.julialang.org/u/Benny)\
**Post date:** [February 10, 2025, 3:59pm UTC](https://discourse.julialang.org/t/julia-is-slower-than-python-when-appending-elements-to-untyped-arrays/125745/32 "2025-02-10T15:59:40Z")

</div>

> [@world-peace](#):
>
> Looking at this code just makes me think all the more that the Julia compiler **must** be producing poor performing or inefficient object code.

We’ve been telling you this the whole thread and explained why.

> [@world-peace](#):
>
> None of you are compiler engineers, so I don’t think your opinions count for much to be honest.

If you only trust the developers of compiler implementations, then why did you even bother asking a wider forum?

> [@world-peace](#):
>
> No, I don’t see why it is fast. Look at all the overhead with the many function calls there to sanity check various conditions during the iteration.

> [@world-peace](#):
>
> Python follows a reference for each element of the list, then does some runtime lookup to see how to add the type of the current value to the type of the current sum value

You’re wrong, look again. If you still don’t get it, keep reading it until you do. If you want help, you can try asking in a Python forum or find the scattered articles and Q&A posts about CPython’s `sum` running faster than naive for-loops.

---

<div class="post-metadata">

**Author:** ![PetrKryslUCSD](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/petrkryslucsd/32/215825_2.png) [@PetrKryslUCSD](https://discourse.julialang.org/u/PetrKryslUCSD)\
**Post date:** [February 10, 2025, 4:09pm UTC](https://discourse.julialang.org/t/julia-is-slower-than-python-when-appending-elements-to-untyped-arrays/125745/33 "2025-02-10T16:09:07Z")

</div>

Has anyone checked the timings?  
I get

```julia
PS src> py .\benchpy.py
starting
finished
time taken: 36871701.0 us
total: 5000000050000000
PS src> julia
               _
   _ _ _(_)_ | Documentation: https://docs.julialang.org
  (_) | (_) (_) |
   _ _ _| |_ __ _ | Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` | |
  | | |_| | | | (_| | | Version 1.10.4 (2024-06-04)
 _/ |\ __'_|_|_|\__'_| | Official https://julialang.org/ release
|__/ |

julia> include("benchju.jl")
starting
finished
time taken: 8.02761e+06 us
total: 5000000050000000

julia> 

```

---

<div class="post-metadata">

**Author:** ![cdawg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cdawg/32/9811_2.png) [@cdawg](https://discourse.julialang.org/u/cdawg)\
**Post date:** [February 10, 2025, 4:12pm UTC](https://discourse.julialang.org/t/julia-is-slower-than-python-when-appending-elements-to-untyped-arrays/125745/34 "2025-02-10T16:12:35Z")

</div>

@world-peace Im curious, whats your motivation here? Do you feel misled by Julias speed claims? Are you angry at the community? Do you want help making your code faster? Or do you just want to complain about this wonderful OSS project, offered as a gift to the world because a julia issue hurt you as a child?

I bet it was closure Box-ing wasnt it ☹ Im sorry buddy, we’ve all been there.

If you think Array{Any} can and should be improved, make a PR. Most of us are happy typing out gigantic arrays, but if you need this, then help out or use a different language. We’re all just bamboozled by your tone here.

---

<div class="post-metadata">

**Author:** ![greatpet](https://avatars.discourse-cdn.com/v4/letter/g/e495f1/32.png) [@greatpet](https://discourse.julialang.org/u/greatpet)\
**Post date:** [February 10, 2025, 4:30pm UTC](https://discourse.julialang.org/t/julia-is-slower-than-python-when-appending-elements-to-untyped-arrays/125745/35 "2025-02-10T16:30:00Z")

</div>

OK, I get it. You want an artificial benchmark that intentionally measures runtime dispatch performance with type-unstable code. Runtime dispatach is well known to be not Julia’s strongest area (small unions are optimizerd; `Any` is not.) But I don’t understand why you’re so upset about the so-called “over-hype” about performance, when no one has been hyping Julia’s performance for non-idiomatic type-unstable code, starting from the manual itself (as I linked above). If you think any part of Julia documentation makes misleading and unrealistic claims about performance without mentioning the usual caveats about type stability etc., you’re welcome to point it out and make a PR.

If you simply want a contrived example showing Julia to be not as optimized as theoretically possible when it “should” be (oh, and one shouldn’t “make excuses”), there is a much easier way: just compare programs with non-const global variables, which can run at full speeds in C and Fortran but incur a penalty in Julia, due to intentional design trade-offs.

---

<div class="post-metadata">

**Author:** ![PetrKryslUCSD](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/petrkryslucsd/32/215825_2.png) [@PetrKryslUCSD](https://discourse.julialang.org/u/PetrKryslUCSD)\
**Post date:** [February 10, 2025, 4:42pm UTC](https://discourse.julialang.org/t/julia-is-slower-than-python-when-appending-elements-to-untyped-arrays/125745/36 "2025-02-10T16:42:31Z")

</div>

And another machine:

```julia
julia> 

julia> include("benchju.jl")
starting
finished
time taken: 4.75440e+06 us
total: 5000000050000000

julia> 
PS rc> py .\benchpy.py
starting
finished
time taken: 9026657.9 us
total: 5000000050000000
PS rc> 

```

So clearly not as reported in the OP.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [February 10, 2025, 4:55pm UTC](https://discourse.julialang.org/t/julia-is-slower-than-python-when-appending-elements-to-untyped-arrays/125745/37 "2025-02-10T16:55:49Z")

</div>

> [@world-peace](#):
>
> This is very similar to what Julia does, except that Python has some overhead due to the fact the function call starts off as Python code before calling into a C level implementation. Since Julia has all the “magic” as others have put it (it isn’t magic) of being JIT-d you would **expect** if efficient machine code was being produced and thus there was none of the overhead which Python suffers from, it **should** be faster but it is not

The overhead of the call is negligible for N=10^8 as in the example here. So you are comparing compiled code to compiled code, but CPython’s implementation has a fast path for adding up lists of numbers that are all the same `int` type (so that it doesn’t need to re-box the intermediate results in new `PyObject`s, and doesn’t need to repeatedly lookup the `+` function).

In principle, Julia’s `sum` function could also add a similar fast path for `Vector{Any}` where it checks that every element is `Int` (or whatever), but it’s probably not worth the tradeoff in code complexity in the stdlib — an idiomatic Julia program won’t use an untyped container in a critical code path.

(Plus, CPython is generally more optimized than Julia for the type-unstable case as has been noted above. In addition to the question of where finite engineering resources should be spent on non-idiomatic code, there are some tradeoffs in reference counting vs. garbage collection for this sort of code that is continually allocating new boxed objects.)

PS. I spent most of a lecture on precisely the example of the `sum` function as [part of MIT’s Performance Engineering class](https://ocw.mit.edu/courses/6-172-performance-engineering-of-software-systems-fall-2018/resources/lecture-23-high-performance-in-dynamic-languages/). It is an interesting case study for why Python’s semantics limit its performance — given `Vector{Any}` semantics, they are doing about as well as it is possible to do, but it is not possible to provide static (compile-time) information about a homogeneous container like `Vector{Int}` or a NumPy array in pure Python.

---

<div class="post-metadata">

**Author:** ![Benny](https://avatars.discourse-cdn.com/v4/letter/b/49beb7/32.png) [@Benny](https://discourse.julialang.org/u/Benny)\
**Post date:** [February 10, 2025, 5:51pm UTC](https://discourse.julialang.org/t/julia-is-slower-than-python-when-appending-elements-to-untyped-arrays/125745/38 "2025-02-10T17:51:45Z")

</div>

> [@PetrKryslUCSD](#):
>
> So clearly not as reported in the OP.

I can corroborate the discrepancy 0.855s \< 3.376s, 2.569s \< 12.269s if uncharging. Not that this thread would last much longer, but I think we can conclude system dependence for the timings. There’s been ample discussion explaining differences in implementation that explains a discrepancy in either direction.

In case anybody wants to compare summation of integer ranges with actual optimization to the point it’s worth using proper benchmarking like `timeit.timeit` and `BenchmarkTools.@btime`, here’s a Python translation of [sum(r::AbstractRange{\<:Real})](https://github.com/JuliaLang/julia/blob/8c62f42947ee01fb17d1baf8c5e953fb3516b31f/base/range.jl#L1421C10-L1421C39):

```julia
def sum_range(x):
  l = len(x)
  return l*x.start + (
         ((x.step * (l-1)) * (l>>1)) if (l%2==0)
    else ((x.step * l) * ((l-1)>>1)) )

```

There’s still overhead that doesn’t get it (206ns) to the same speed (2ns) as Julia (`int` doesn’t overflow, it’s pretty sophisticated design to optimize smaller values even this much), but it’s much faster than the 1.3s that `sum(range(1,100000001))` takes, which is nonideally even slower than summing the full `list`, and also puts `numpy.arange` to shame. Moral of the story is to use the right types and algorithms before pointing fingers at a compiler or lack thereof.

---

<div class="post-metadata">

**Author:** ![Mason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mason/32/2423_2.png) [@Mason](https://discourse.julialang.org/u/Mason)\
**Post date:** [February 10, 2025, 6:19pm UTC](https://discourse.julialang.org/t/julia-is-slower-than-python-when-appending-elements-to-untyped-arrays/125745/39 "2025-02-10T18:19:34Z")

</div>

> [@world-peace](#):
>
> Python takes about 3 seconds on my machine. Julia takes about 20 seconds.
> 
> Is there any particular reason for this or is it time to admit the over-hyped claims made about Julia’s performance are blatantly false?
> 
> Close to C performance? I don’t think so.

Sufficiently poorly written code can be arbitrarily slow in any language.

If you’d like some help, I can help you write an even slower version in C.

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [February 10, 2025, 6:48pm UTC](https://discourse.julialang.org/t/julia-is-slower-than-python-when-appending-elements-to-untyped-arrays/125745/40 "2025-02-10T18:48:25Z")

</div>

> [@world-peace](#):
>
> What this shows is - for whatever reason - the performance of Julia is nothing like what it is claimed to be.

As far as I can tell, your benchmark beautifully demonstrates that the performance of Julia is _exactly_ like what it is claimed to be.

[Previous page](https://discourse.julialang.org/t/julia-is-slower-than-python-when-appending-elements-to-untyped-arrays/125745.md?page=1)

[Next page](https://discourse.julialang.org/t/julia-is-slower-than-python-when-appending-elements-to-untyped-arrays/125745.md?page=3)