# Need help on understanding type inference / performance

**URL:** <https://discourse.julialang.org/t/need-help-on-understanding-type-inference-performance/80600>\
**Category:** Performance\
**Tags:** question, inference\
**Created:** [May 6, 2022, 10:55am UTC](https://discourse.julialang.org/t/need-help-on-understanding-type-inference-performance/80600 "2022-05-06T10:55:04Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![schlichtanders](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/schlichtanders/32/32145_2.png) [@schlichtanders](https://discourse.julialang.org/u/schlichtanders)\
**Post date:** [May 6, 2022, 10:55am UTC](https://discourse.julialang.org/t/need-help-on-understanding-type-inference-performance/80600/1 "2022-05-06T10:55:04Z")

</div>

Hi there,

I am the author of [ExtensibleEffects.jl](https://github.com/JuliaFunctional/ExtensibleEffects.jl) and several people approached me that it would be great to improve performance (it is currently super slow).

I made a first attempt on inspecting and found the following specific case to be one crucial component

```julia
struct ChainedFunctions{Fs}
    functions::Fs
    ChainedFunctions(functions...) = new{typeof(functions)}(functions)
end

instantiate(T) = T()

function instantiate_wrapper1(type)
    continuation = ChainedFunctions(() -> instantiate(type))
    first_func = Base.first(continuation.functions)
    first_func()
end

function instantiate_wrapper2(::Type{type}) where type
    continuation = ChainedFunctions(() -> instantiate(type))
    first_func = Base.first(continuation.functions)
    first_func()
end

using Test
@inferred instantiate_wrapper1(Vector) 
# ERROR: return type Vector{Any} does not match inferred return type Any
@inferred instantiate_wrapper2(Vector)
# Any[]

```

This is run on julia 1.7.1. It is quite a pity, as to the best of my understanding it should never make a type-inference difference in normal functions whether you use `type::Type` or `::Type{type} where type`.

Furthermore, this distinction makes it impossible to write generic code, as out of a sudden, passing a type like `Vector` is something completely different than passing a concrete struct instance.

Motivation: In ExtensibleEffects `Vector` and others are `effect-handlers`. While you can have a generic effect handler implementation, like indeed for `Vector`, there may be the need for extra information in order to run the handler. The latter requires a concrete handler struct which captures those extra information. Both kinds of effect handlers are very intuitive and I would not like fallback to something like `Val{Vector}()` to support `Vector` - it would complicate the entire interface at many places.

* * *

Any help is highly appreciated. If someone knows, why this is going on, or how a general workaround could look like, or whether this may be unintended and worth a feature request, or even a bug report, everything is very much welcome.

---

<div class="post-metadata">

**Author:** ![schlichtanders](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/schlichtanders/32/32145_2.png) [@schlichtanders](https://discourse.julialang.org/u/schlichtanders)\
**Post date:** [May 6, 2022, 10:58am UTC](https://discourse.julialang.org/t/need-help-on-understanding-type-inference-performance/80600/2 "2022-05-06T10:58:46Z")

</div>

BAM! I just now imagined the workaround:

```julia
function instantiate_wrapper3(::Type{type}) where type
    instantiate_wrapper1(type)
end
@inferred instantiate_wrapper3(Vector)
# Any[]

```

Very surprising, that you have to specify such a fallback for type-inference to work correctly.  
For me it looks like this wrapper shouldn’t be necessary.

EDIT: this workaround apparently fails very quickly. See my below reply

---

<div class="post-metadata">

**Author:** ![jakobnissen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jakobnissen/32/13477_2.png) [@jakobnissen](https://discourse.julialang.org/u/jakobnissen)\
**Post date:** [May 6, 2022, 11:26am UTC](https://discourse.julialang.org/t/need-help-on-understanding-type-inference-performance/80600/3 "2022-05-06T11:26:15Z")

</div>

It’s presumably related to this:  
[https://docs.julialang.org/en/v1/manual/performance-tips/#Be-aware-of-when-Julia-avoids-specializing](https://docs.julialang.org/en/v1/manual/performance-tips/#Be-aware-of-when-Julia-avoids-specializing)

I.e. Julia intentionally does not specialize `instantiate_wrapper1` because the input type is not used directly in the function. However, I do agree that this behaviour is a bit of a footgun, or at least can be perplexing.

---

<div class="post-metadata">

**Author:** ![schlichtanders](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/schlichtanders/32/32145_2.png) [@schlichtanders](https://discourse.julialang.org/u/schlichtanders)\
**Post date:** [May 6, 2022, 11:49am UTC](https://discourse.julialang.org/t/need-help-on-understanding-type-inference-performance/80600/4 "2022-05-06T11:49:34Z")

</div>

thank you for the reference.

can you explain, why the wrapper is enough to make it work? The inner function should still not specialize, or should it?

---

<div class="post-metadata">

**Author:** ![schlichtanders](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/schlichtanders/32/32145_2.png) [@schlichtanders](https://discourse.julialang.org/u/schlichtanders)\
**Post date:** [May 6, 2022, 2:15pm UTC](https://discourse.julialang.org/t/need-help-on-understanding-type-inference-performance/80600/5 "2022-05-06T14:15:27Z")

</div>

Unfortunately, this workaround seems to be a special case and already does not work any longer for two arguments

```julia
singleton_array(T::Type{<:AbstractArray}, a) = convert(T, [a]) 

struct ChainedFunctions{Fs}
    functions::Fs
    ChainedFunctions(functions...) = new{typeof(functions)}(functions)
end

function singleton_wrapper1(type, value)
    continuation = ChainedFunctions(x -> singleton_array(type, x))
    first_func = Base.first(continuation.functions)
    first_func(value)
end

function singleton_wrapper2(type::T, value) where T
    continuation = ChainedFunctions(x -> singleton_array(type, x))
    first_func = Base.first(continuation.functions)
    first_func(value)
end

function singleton_wrapper3(::Type{type}, value) where type
    continuation = ChainedFunctions(x -> singleton_array(type, x))
    first_func = Base.first(continuation.functions)
    first_func(value)
end

function singleton_wrapper1_wrapper1(::Type{type}, value) where type
    singleton_wrapper1(type, value)
end
function singleton_wrapper1_wrapper1(type, value)
    singleton_wrapper1(type, value)
end

@inferred singleton_wrapper1(Vector, 1)
# ERROR: return type Vector{Int64} does not match inferred return type Any
@inferred singleton_wrapper2(Vector, 1)
# ERROR: return type Vector{Int64} does not match inferred return type Any
@inferred singleton_wrapper3(Vector, 1)
# [1]

@inferred singleton_wrapper1_wrapper1(Vector, 1)
# ERROR: return type Vector{Int64} does not match inferred return type Any

```

How can I trigger the specialization on `Type{T}` while still keeping the generic function which works on other inputs than Type?

---

<div class="post-metadata">

**Author:** ![Henrique\_Becker](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/henrique_becker/32/15443_2.png) [@Henrique\_Becker](https://discourse.julialang.org/u/Henrique_Becker)\
**Post date:** [May 6, 2022, 5:51pm UTC](https://discourse.julialang.org/t/need-help-on-understanding-type-inference-performance/80600/6 "2022-05-06T17:51:49Z")

</div>

Can you explain a little better what your code intend to do? It seems to me that you have a `struct` for which your type parameter is often a `Vector` containing multiple distinct functions, and in your you retrieve one (the first?) of these function and apply to another input, how exactly you expect Julia to infer anything? It seems magical to me that Julia is able to infer anything at all, and if it does is probably because constant propagation, no?

---

<div class="post-metadata">

**Author:** ![goerch](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/goerch/32/29122_2.png) [@goerch](https://discourse.julialang.org/u/goerch)\
**Post date:** [May 6, 2022, 6:28pm UTC](https://discourse.julialang.org/t/need-help-on-understanding-type-inference-performance/80600/7 "2022-05-06T18:28:51Z")

</div>

> [@schlichtanders](#):
>
> How can I trigger the specialization on `Type{T}` while still keeping the generic function which works on other inputs than Type?

I’m not sure I understand what you are doing, but the following seems to work for me on Julia 1.9.0

```julia
singleton_array(T::Type{<:AbstractArray}, a) = convert(T, [a]) 

struct ChainedFunctions{Fs}
    functions::Fs
    ChainedFunctions(functions...) = new{typeof(functions)}(functions)
end

function singleton_wrapper1(type, value)
    continuation = ChainedFunctions(x -> singleton_array(type, x))
    first_func = Base.first(continuation.functions)
    first_func(value)
end

function singleton_wrapper2(type::T, value) where T
    continuation = ChainedFunctions(x -> singleton_array(T, x))
    first_func = Base.first(continuation.functions)
    first_func(value)
end

function singleton_wrapper3(::Type{T}, value) where T
    continuation = ChainedFunctions(x -> singleton_array(T, x))
    first_func = Base.first(continuation.functions)
    first_func(value)
end

function singleton_wrapper_wrapper1(::Type{T}, value) where T
    singleton_wrapper1(T, value)
end
function singleton_wrapper_wrapper1(type, value)
    singleton_wrapper1(type, value)
end

function singleton_wrapper_wrapper3(::Type{T}, value) where T
    singleton_wrapper3(T, value)
end
function singleton_wrapper_wrapper3(type, value)
    singleton_wrapper3(type, value)
end

using Test
# @inferred singleton_wrapper1(Vector, 1) # ERROR: return type Vector{Int64} does not match inferred return type Any
# @inferred singleton_wrapper2(Vector, 1) #ERROR: MethodError: no method matching singleton_array(::Type{UnionAll}, ::Int64)
@inferred singleton_wrapper3(Vector, 1) #works
# @inferred singleton_wrapper_wrapper1(Vector, 1) # ERROR: return type Vector{Int64} does not match inferred return type Any
@inferred singleton_wrapper_wrapper3(Vector, 1)

```

---

<div class="post-metadata">

**Author:** ![schlichtanders](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/schlichtanders/32/32145_2.png) [@schlichtanders](https://discourse.julialang.org/u/schlichtanders)\
**Post date:** [May 7, 2022, 7:16am UTC](https://discourse.julialang.org/t/need-help-on-understanding-type-inference-performance/80600/8 "2022-05-07T07:16:16Z")

</div>

Julia can infer everything if it specializes the function also over types.

The type parameter does not contain a `Vector` but a `Tuple` which indeed captures all relevant type information. The only type-information which is lost is that of the `handler`, and that gets lost because of missing specialization apparently.

---

<div class="post-metadata">

**Author:** ![schlichtanders](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/schlichtanders/32/32145_2.png) [@schlichtanders](https://discourse.julialang.org/u/schlichtanders)\
**Post date:** [May 7, 2022, 7:18am UTC](https://discourse.julialang.org/t/need-help-on-understanding-type-inference-performance/80600/9 "2022-05-07T07:18:19Z")

</div>

Hi @goerch thank you for testing on 1.9

I get that `singleton_wrapper_wrapper3` works on 1.9, which sounds great. Would be nice to have it for 1.6 so that I could support julia’s longterm release.  
Thank you for your help.

---

<div class="post-metadata">

**Author:** ![schlichtanders](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/schlichtanders/32/32145_2.png) [@schlichtanders](https://discourse.julialang.org/u/schlichtanders)\
**Post date:** [May 7, 2022, 7:36am UTC](https://discourse.julialang.org/t/need-help-on-understanding-type-inference-performance/80600/10 "2022-05-07T07:36:35Z")

</div>

Thank you all for your help. It seems the reason is understood: Missing type-specialization is the reason.

I still struggle with how to enable this and created a more concrete, hopefully also clearer, follow up question: [How to mark a function argument for specialization on types while still staying as a generic argument?](https://discourse.julialang.org/t/how-to-mark-a-function-argument-for-specialization-on-types-while-still-staying-as-a-generic-argument/80652)

---

<div class="post-metadata">

**Author:** ![aviatesk](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/aviatesk/32/7610_2.png) [@aviatesk](https://discourse.julialang.org/u/aviatesk)\
**Post date:** [May 7, 2022, 7:55am UTC](https://discourse.julialang.org/t/need-help-on-understanding-type-inference-performance/80600/11 "2022-05-07T07:55:21Z")

</div>

Is it really necessary to work on `Type{...}`-signatures? It can sometimes confuse inference very much especially and thus should generally be avoided if possible as described in the performance tip @jakobnissen posted above. A note here would be that additional type parameter causes specialization by _giving up inference_ and resorting to dynamic dispatch.

---

<div class="post-metadata">

**Author:** ![schlichtanders](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/schlichtanders/32/32145_2.png) [@schlichtanders](https://discourse.julialang.org/u/schlichtanders)\
**Post date:** [May 7, 2022, 8:31am UTC](https://discourse.julialang.org/t/need-help-on-understanding-type-inference-performance/80600/12 "2022-05-07T08:31:02Z")

</div>

I wrote my code without the `Type{...}` signature which unfortunately results in very slow performance.

as @jakobnissen pointed out, this is due to Julia compiler **requiring** the `Type{...}` signature. It is not me who makes this necessary, it is the current Julia compiler.

---

<div class="post-metadata">

**Author:** ![Henrique\_Becker](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/henrique_becker/32/15443_2.png) [@Henrique\_Becker](https://discourse.julialang.org/u/Henrique_Becker)\
**Post date:** [May 7, 2022, 1:30pm UTC](https://discourse.julialang.org/t/need-help-on-understanding-type-inference-performance/80600/13 "2022-05-07T13:30:25Z")

</div>

Ah, ok, it did not become clear for me anywhere that it was a tuple. Yes, if it is a tuple then the type of the elements is not simplified to a common denominator and this can work.

---

<div class="post-metadata">

**Author:** ![schlichtanders](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/schlichtanders/32/32145_2.png) [@schlichtanders](https://discourse.julialang.org/u/schlichtanders)\
**Post date:** [May 10, 2022, 9:54am UTC](https://discourse.julialang.org/t/need-help-on-understanding-type-inference-performance/80600/14 "2022-05-10T09:54:29Z")

</div>

Thank you all for your input

I filed a proper report at [Bug Report: Type argument not specializing for closures · Issue #45257 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/issues/45257)