# Why is this code run-time dispatch/slow?

**URL:** <https://discourse.julialang.org/t/why-is-this-code-run-time-dispatch-slow/47178>\
**Category:** Performance\
**Created:** [September 24, 2020, 5:24am UTC](https://discourse.julialang.org/t/why-is-this-code-run-time-dispatch-slow/47178 "2020-09-24T05:24:24Z")\
**Posts on this page:** 17\
**Page:** 1

<div class="post-metadata">

**Author:** ![JamesK](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jamesk/32/8989_2.png) [@JamesK](https://discourse.julialang.org/u/JamesK)\
**Post date:** [September 24, 2020, 5:24am UTC](https://discourse.julialang.org/t/why-is-this-code-run-time-dispatch-slow/47178/1 "2020-09-24T05:24:24Z")

</div>

I made a minimum reproducible example for a performance issue I am having with my code:

```julia
mutable struct Array_container
    array_field::Array{Float64}
end

function foo(a::Array_container, b::Array_container)
    a.array_field[1] - b.array_field[1]
end

function run_it_n_times(n::Int64)
    a = Array_container([1.1])
    b = Array_container([2.2])
    for i in (1:n)
        foo(a, b)
    end
end

```

I run

> @profview run\_it\_n\_times(10000000)

and observe that the function foo() is “run-time dispatch” since it’s in red (source: [GitHub - timholy/ProfileView.jl: Visualization of Julia profiling data](https://github.com/timholy/ProfileView.jl)). See below for the image.

Specifically, the docs say that it is “dynamic dispatch, run-time method lookup, or a virtual call”. I don’t see any obvious type instability (I am a beginner though), or any other issues addressed in the Performance Tips section of the manual. What is wrong with this code and how do I fix its performance? Thank you!

 ![image](https://global.discourse-cdn.com/julialang/original/3X/4/a/4aeecaf91641982918f9a1cc40f80cad38d7d817.png)

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [September 24, 2020, 5:33am UTC](https://discourse.julialang.org/t/why-is-this-code-run-time-dispatch-slow/47178/2 "2020-09-24T05:33:37Z")

</div>

I think what the red in this case is the allocation of an array to store the result of `foo`, not any runtime dispatch.

---

<div class="post-metadata">

**Author:** ![JamesK](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jamesk/32/8989_2.png) [@JamesK](https://discourse.julialang.org/u/JamesK)\
**Post date:** [September 24, 2020, 5:39am UTC](https://discourse.julialang.org/t/why-is-this-code-run-time-dispatch-slow/47178/3 "2020-09-24T05:39:26Z")

</div>

> [@Oscar\_Smith](#):
>
> I think what the red in this case is the allocation of an array to store the result of `foo`, not any runtime dispatch.

Thank you for the reply! I don’t think that’s the problem in this case. The following code does not show those red bars:

```julia
function foobar(a::Array{Float64}, b::Array{Float64})
    a[1] - b[1]
end

function run_foobar_n_times(n::Int64)
    for i in (1:n)
        foobar([1.1], [2.2])
    end
end

```

---

<div class="post-metadata">

**Author:** ![remi-garcia](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/remi-garcia/32/7125_2.png) [@remi-garcia](https://discourse.julialang.org/u/remi-garcia)\
**Post date:** [September 24, 2020, 6:01am UTC](https://discourse.julialang.org/t/why-is-this-code-run-time-dispatch-slow/47178/4 "2020-09-24T06:01:35Z")

</div>

`@benchmark` to illustrate this:

```nohighlight
julia> using BenchmarkTools

julia> mutable struct Array_container
           array_field::Array{Float64}
       end

julia> function foo(a::Array_container, b::Array_container)
           a.array_field[1] - b.array_field[1]
       end;

julia> function bar(a::Array{Float64}, b::Array{Float64})
           a[1] - b[1]
       end;

julia> @benchmark foo(a, b) setup=((a,b)=(Array_container([rand()]), Array_container([rand()])))
BenchmarkTools.Trial: 
  memory estimate: 32 bytes
  allocs estimate: 2
  --------------
  minimum time: 50.809 ns (0.00% GC)
  median time: 51.581 ns (0.00% GC)
  mean time: 61.539 ns (11.28% GC)
  maximum time: 47.769 μs (99.86% GC)
  --------------
  samples: 10000
  evals/sample: 987

julia> @benchmark bar(a, b) setup=((a,b)=([rand()], [rand()]))
BenchmarkTools.Trial: 
  memory estimate: 0 bytes
  allocs estimate: 0
  --------------
  minimum time: 2.211 ns (0.00% GC)
  median time: 2.233 ns (0.00% GC)
  mean time: 2.273 ns (0.00% GC)
  maximum time: 30.894 ns (0.00% GC)
  --------------
  samples: 10000
  evals/sample: 1000

```

---

<div class="post-metadata">

**Author:** ![remi-garcia](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/remi-garcia/32/7125_2.png) [@remi-garcia](https://discourse.julialang.org/u/remi-garcia)\
**Post date:** [September 24, 2020, 6:08am UTC](https://discourse.julialang.org/t/why-is-this-code-run-time-dispatch-slow/47178/5 "2020-09-24T06:08:31Z")

</div>

I don’t know if that would be called type instability but:

```nohighlight
julia> Array{Float64}
Array{Float64,N} where N

julia> Array{Float64, 1}
Array{Float64,1}

```

Precising the dim of `array_field` inside the `mutable struct` makes a great difference (but I have no explication, sorry):

```nohighlight
julia> using BenchmarkTools

julia> mutable struct Array_container
           array_field::Array{Float64, 1}
       end

julia> function foo(a::Array_container, b::Array_container)
           a.array_field[1] - b.array_field[1]
       end;

julia> function bar(a::Array{Float64, 1}, b::Array{Float64, 1})
           a[1] - b[1]
       end;

julia> @benchmark foo(a, b) setup=((a,b)=(Array_container([rand()]), Array_container([rand()])))
BenchmarkTools.Trial: 
  memory estimate: 0 bytes
  allocs estimate: 0
  --------------
  minimum time: 4.130 ns (0.00% GC)
  median time: 4.181 ns (0.00% GC)
  mean time: 4.286 ns (0.00% GC)
  maximum time: 50.718 ns (0.00% GC)
  --------------
  samples: 10000
  evals/sample: 1000

julia> @benchmark bar(a, b) setup=((a,b)=([rand()], [rand()]))
BenchmarkTools.Trial: 
  memory estimate: 0 bytes
  allocs estimate: 0
  --------------
  minimum time: 2.641 ns (0.00% GC)
  median time: 2.656 ns (0.00% GC)
  mean time: 2.808 ns (0.00% GC)
  maximum time: 62.966 ns (0.00% GC)
  --------------
  samples: 10000
  evals/sample: 1000

```

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [September 24, 2020, 6:42am UTC](https://discourse.julialang.org/t/why-is-this-code-run-time-dispatch-slow/47178/6 "2020-09-24T06:42:07Z")

</div>

Abstract field types in structs are a performance trap. Use concrete, possibly parameterized, types for the fields.

Also, do you need `Array_container` to be a mutable struct? That could also affect performance. Since the wrapped array is already mutable, you don’t need the outer type to be mutable also, in most cases.

BTW, the convention for type names is to use CapitalCase not underscores, so `ArrayContainer`.

---

<div class="post-metadata">

**Author:** ![tisztamo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tisztamo/32/16200_2.png) [@tisztamo](https://discourse.julialang.org/u/tisztamo)\
**Post date:** [September 24, 2020, 6:54am UTC](https://discourse.julialang.org/t/why-is-this-code-run-time-dispatch-slow/47178/7 "2020-09-24T06:54:26Z")

</div>

> [@remi-garcia](#):
>
> Precising the dim of `array_field` inside the `mutable struct` makes a great difference (but I have no explication, sorry):

Without fully parametrizing the Array, it remains a UnionAll type, which is not concrete (although the [performance tips in the docs](https://docs.julialang.org/en/v1/manual/performance-tips/#Avoid-fields-with-abstract-containers) speaks about abstract types, I think non-concrete would be a better term)

The explanation is that the compiler cannot know the memory layout of a UnionAll type, so it has to compile code for checking the runtime type (which is concrete). That check is the dynamic dispatch in the OP.

You can use `Vector{Float64}` for one-dimensional arrays, it looks nicer.

---

<div class="post-metadata">

**Author:** ![JamesK](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jamesk/32/8989_2.png) [@JamesK](https://discourse.julialang.org/u/JamesK)\
**Post date:** [September 25, 2020, 4:55am UTC](https://discourse.julialang.org/t/why-is-this-code-run-time-dispatch-slow/47178/8 "2020-09-25T04:55:19Z")

</div>

Thank you very much for the answers and explanations, my Julia codebase performance finally matches that of C++!

Should something along these lines be put in the [Avoid fields with abstract containers](https://docs.julialang.org/en/v1/manual/performance-tips/#Avoid-fields-with-abstract-containers) section of the docs? The docs skirt this issue, but seem to never directly address it (unless if I’m missing it), and other beginners may have similar points of confusion.

Perhaps the simple addition of the following to that section of the docs:  
=============================

```julia
struct MyContainer
    a::Array{Float64}
end

```

is bad: it’s still an abstract type since the dimensionality is not specified. Instead, consider using something like:

```julia
struct MyContainer
    a::Array{Float64, 1}
end

```

(replace 1 with n for n-dimensions) or

```julia
struct MySimpleContainer{A<:AbstractVector}
    a::A
end

```

=============================  
This addition to that section may help a lot of people. In my case, it would have saved me quite a few hours.

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [September 25, 2020, 5:08am UTC](https://discourse.julialang.org/t/why-is-this-code-run-time-dispatch-slow/47178/9 "2020-09-25T05:08:07Z")

</div>

Our performance tips should probably mention that `Array` isn’t `Vector` and that can be a performance issue. I’m not sure why but a ton of newcomers use `Array` when they really want `Vector` or `Matrix`. Maybe it’s from Matlab where those aren’t distinct?

---

<div class="post-metadata">

**Author:** ![tbeason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tbeason/32/15898_2.png) [@tbeason](https://discourse.julialang.org/u/tbeason)\
**Post date:** [September 25, 2020, 5:10am UTC](https://discourse.julialang.org/t/why-is-this-code-run-time-dispatch-slow/47178/10 "2020-09-25T05:10:14Z")

</div>

> [@Oscar\_Smith](#):
>
> Maybe it’s from Matlab where those aren’t distinct?

I think that and also that `Array` is typically what gets printed at the REPL.

---

<div class="post-metadata">

**Author:** ![JamesK](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jamesk/32/8989_2.png) [@JamesK](https://discourse.julialang.org/u/JamesK)\
**Post date:** [September 25, 2020, 5:14am UTC](https://discourse.julialang.org/t/why-is-this-code-run-time-dispatch-slow/47178/11 "2020-09-25T05:14:07Z")

</div>

> [@Oscar\_Smith](#):
>
> Our performance tips should probably mention that `Array` isn’t `Vector` and that can be a performance issue. I’m not sure why but a ton of newcomers use `Array` when they really want `Vector` or `Matrix` . Maybe it’s from Matlab where those aren’t distinct?

Huh, I just tried replacing all Array{Float64, 1} with Vector{Float64} and it’s not making any noticeable performance difference in my case. What really helped was specifying the dimension. This could just be my use case though, perhaps in other cases using Vector instead makes a big difference?

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [September 25, 2020, 5:15am UTC](https://discourse.julialang.org/t/why-is-this-code-run-time-dispatch-slow/47178/12 "2020-09-25T05:15:10Z")

</div>

`Vector{T}` is an alias for `Array{T,1}`. They are exactly the same. The problem is only when using `Array` makes you forget a parameter and therefore have non-static types.

---

<div class="post-metadata">

**Author:** ![paulmelis](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/paulmelis/32/35063_2.png) [@paulmelis](https://discourse.julialang.org/u/paulmelis)\
**Post date:** [September 25, 2020, 7:05am UTC](https://discourse.julialang.org/t/why-is-this-code-run-time-dispatch-slow/47178/13 "2020-09-25T07:05:48Z")

</div>

> [@Oscar\_Smith](#):
>
> `Vector{T}` is an alias for `Array{T,1}` . They are exactly the same. The problem is only when using `Array` makes you forget a parameter and therefore have non-static types.

Slightly off-topic, but I was digging through the Base sources to figure out how `Array` is defined exactly to further understand the behaviour above. But `base/array.jl` does not contain a definition of `Array` as far as I can tell, nor do any of the other julia source files. I see the `Array` type being referenced at the top of `base/Base.jl`, even before `base/array.jl` is included. Is the `Array` type defined in C code?

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [September 25, 2020, 7:26am UTC](https://discourse.julialang.org/t/why-is-this-code-run-time-dispatch-slow/47178/14 "2020-09-25T07:26:10Z")

</div>

> [@paulmelis](#):
>
> Is the `Array` type defined in C code?

[I think so, yes.](https://github.com/JuliaLang/julia/blob/master/src/array.c) It’s one of the few things defined that way, possibly because arrays are needed very early on in the chain, when almost none of the machinery for calling and compiling julia exists.

---

<div class="post-metadata">

**Author:** ![paulmelis](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/paulmelis/32/35063_2.png) [@paulmelis](https://discourse.julialang.org/u/paulmelis)\
**Post date:** [September 25, 2020, 7:52am UTC](https://discourse.julialang.org/t/why-is-this-code-run-time-dispatch-slow/47178/15 "2020-09-25T07:52:56Z")

</div>

> [@Oscar\_Smith](#):
>
> The problem is only when using `Array` makes you forget a parameter and therefore have non-static types.

Coming from C++ this is a real mind-bender. The conceptually equivalent construct in C++ would be a template `Array<T,N>`, but there is no way to reference `Array` _as a type_ without filling in all of the parameters.

---

<div class="post-metadata">

**Author:** ![remi-garcia](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/remi-garcia/32/7125_2.png) [@remi-garcia](https://discourse.julialang.org/u/remi-garcia)\
**Post date:** [September 25, 2020, 9:53am UTC](https://discourse.julialang.org/t/why-is-this-code-run-time-dispatch-slow/47178/16 "2020-09-25T09:53:16Z")

</div>

On a project with friends we talk about the _style_ that we will use for the code and most of them wanted `Array{Float64, 1}` over `Vector{Float64}`. I think it is because they are use to write `Array{Float64, 1}` in Julia as they started this way when they did not know the existence of `Vector`.

---

<div class="post-metadata">

**Author:** ![Elrod](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/elrod/32/22461_2.png) [@Elrod](https://discourse.julialang.org/u/Elrod)\
**Post date:** [September 25, 2020, 10:34am UTC](https://discourse.julialang.org/t/why-is-this-code-run-time-dispatch-slow/47178/17 "2020-09-25T10:34:29Z")

</div>

Among the many cool new things in Julia master:

```julia
julia> rand(3)
3-element Vector{Float64}:
 0.24215660775997416
 0.5386524854737231
 0.5880707121465651

julia> typeof(ans)
Vector{Float64} (alias for Array{Float64, 1})

```