# Force specialization on kwargs

**URL:** <https://discourse.julialang.org/t/force-specialization-on-kwargs/34789>\
**Category:** Performance\
**Tags:** kwargs\
**Created:** [February 18, 2020, 7:04am UTC](https://discourse.julialang.org/t/force-specialization-on-kwargs/34789 "2020-02-18T07:04:34Z")\
**Posts on this page:** 16\
**Page:** 1

<div class="post-metadata">

**Author:** ![dilumaluthge](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dilumaluthge/32/29283_2.png) [@dilumaluthge](https://discourse.julialang.org/u/dilumaluthge)\
**Post date:** [February 18, 2020, 7:04am UTC](https://discourse.julialang.org/t/force-specialization-on-kwargs/34789/1 "2020-02-18T07:04:34Z")

</div>

If I have a function like this:

```julia
f_vararg(varargs::Int...) = tuple(varargs...)

```

I can force specialization on varargs with either of the following:

```julia
g_vararg(varargs::Vararg{Int, N}) where {N} = tuple(varargs...)

```

```julia
h_vararg(varargs::Vararg{<:Any, N}) where {N} = tuple(varargs...)

```

* * *

Now suppose instead that I have a function like this:

```julia
f_kwarg(; kwargs...) = (; kwargs...)

```

How can I force specialization on the keyword arguments?

---

<div class="post-metadata">

**Author:** ![tkf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkf/32/17635_2.png) [@tkf](https://discourse.julialang.org/u/tkf)\
**Post date:** [February 18, 2020, 7:39am UTC](https://discourse.julialang.org/t/force-specialization-on-kwargs/34789/2 "2020-02-18T07:39:43Z")

</div>

How do you check specialization of functions with keyword arguments? Does this show that apparently they are automatically specialized?

```julia
julia> f_kwarg(; kwargs...) = (; kwargs...)
f_kwarg (generic function with 1 method)

julia> (@which f_kwarg(a=1, b=2.0, c=:three)).specializations

julia> @code_lowered f_kwarg(a=1, b=2.0, c=:three)
CodeInfo(
1 ─ kwargs... = Base.pairs(@_2)
│ %2 = Main.:(var"#f_kwarg#3")(kwargs..., @_3)
└── return %2
)

julia> (@which var"#f_kwarg#3"((a=1, b=2.0, c=:three), f_kwarg)).specializations

julia> f_kwarg(a=1, b=2.0, c=:three)
(a = 1, b = 2.0, c = :three)

julia> (@which f_kwarg(a=1, b=2.0, c=:three)).specializations
Core.TypeMapEntry(nothing, Tuple{var"#f_kwarg##kw",NamedTuple{(:a, :b, :c),Tuple{Int64,Float64,Symbol}},typeof(f_kwarg)}, nothing, svec(), 0x0000000000000001, 0xffffffffffffffff, MethodInstance for (::var"#f_kwarg##kw")(::NamedTuple{(:a, :b, :c),Tuple{Int64,Float64,Symbol}}, ::typeof(f_kwarg)), true, true, false)

julia> (@which var"#f_kwarg#3"((a=1, b=2.0, c=:three), f_kwarg)).specializations
Core.TypeMapEntry(nothing, Tuple{var"##f_kwarg#3",Base.Iterators.Pairs{Symbol,Any,Tuple{Symbol,Symbol,Symbol},NamedTuple{(:a, :b, :c),Tuple{Int64,Float64,Symbol}}},typeof(f_kwarg)}, nothing, svec(), 0x0000000000000001, 0xffffffffffffffff, MethodInstance for #f_kwarg#3(::Base.Iterators.Pairs{Symbol,Any,Tuple{Symbol,Symbol,Symbol},NamedTuple{(:a, :b, :c),Tuple{Int64,Float64,Symbol}}}, ::typeof(f_kwarg)), true, true, false)

```

---

<div class="post-metadata">

**Author:** ![JeffreySarnoff](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jeffreysarnoff/32/1980_2.png) [@JeffreySarnoff](https://discourse.julialang.org/u/JeffreySarnoff)\
**Post date:** [February 18, 2020, 7:44am UTC](https://discourse.julialang.org/t/force-specialization-on-kwargs/34789/3 "2020-02-18T07:44:26Z")

</div>

Julia does not [yet] specialize dispatch on keyword arguments.

---

<div class="post-metadata">

**Author:** ![dilumaluthge](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dilumaluthge/32/29283_2.png) [@dilumaluthge](https://discourse.julialang.org/u/dilumaluthge)\
**Post date:** [February 18, 2020, 7:45am UTC](https://discourse.julialang.org/t/force-specialization-on-kwargs/34789/4 "2020-02-18T07:45:02Z")

</div>

I don’t think that we can trust the output of `@code_lowered`, etc when it comes to specialization. See e.g. [Can we support a way to make `@code_typed` stop lying to us all? :) · Issue #32834 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/issues/32834) and [https://github.com/JuliaLang/julia/blob/master/doc/src/manual/performance-tips.md:](https://github.com/JuliaLang/julia/blob/master/doc/src/manual/performance-tips.md:)

> Note that `@code_typed` and friends will always show you specialized code, even if Julia would not normally specialize that method call.

---

<div class="post-metadata">

**Author:** ![dilumaluthge](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dilumaluthge/32/29283_2.png) [@dilumaluthge](https://discourse.julialang.org/u/dilumaluthge)\
**Post date:** [February 18, 2020, 7:48am UTC](https://discourse.julialang.org/t/force-specialization-on-kwargs/34789/5 "2020-02-18T07:48:15Z")

</div>

> [@JeffreySarnoff](#):
>
> Julia does not [yet] specialize dispatch on keyword arguments.

Well, dispatch is a different question entirely. Keyword arguments do not participate in dispatch. But I’m asking if I can get Julia to compile specialized code for the types of the keyword arguments.

If I understand correctly, specialization on keyword arguments would not require keyword arguments to participate in dispatch.

---

<div class="post-metadata">

**Author:** ![tkf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkf/32/17635_2.png) [@tkf](https://discourse.julialang.org/u/tkf)\
**Post date:** [February 18, 2020, 7:56am UTC](https://discourse.julialang.org/t/force-specialization-on-kwargs/34789/7 "2020-02-18T07:56:48Z")

</div>

I’m using `@code_lowered` just to get the name of the “body function” `var"#f_kwarg#3"` as it looks like `(@which f_kwarg(a=1, b=2.0, c=:three)).specializations` is showing the specializations of the “kwsorter” function.

In that REPL session, AFAICT both keyword sorter and the body function are specialized (as I see `Tuple{Int64,Float64,Symbol}`). But I don’t know if this is the right way to do it.

---

<div class="post-metadata">

**Author:** ![dilumaluthge](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dilumaluthge/32/29283_2.png) [@dilumaluthge](https://discourse.julialang.org/u/dilumaluthge)\
**Post date:** [February 18, 2020, 7:59am UTC](https://discourse.julialang.org/t/force-specialization-on-kwargs/34789/8 "2020-02-18T07:59:08Z")

</div>

Yeah, it certainly does **seem** as if specialization is happening.

I am skeptical though. If specialization does not happen by default on varargs (thus requiring us to add the `Vararg` type parameter), why would it happen by default on keyword arguments?

I think the more likely situation is that we are seeing misleading output, and specialization is not actually happening here.

---

<div class="post-metadata">

**Author:** ![tkf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkf/32/17635_2.png) [@tkf](https://discourse.julialang.org/u/tkf)\
**Post date:** [February 18, 2020, 8:13am UTC](https://discourse.julialang.org/t/force-specialization-on-kwargs/34789/9 "2020-02-18T08:13:30Z")

</div>

Yeah, I agree it’s strange and that’s why I feel like I’m missing something.

But function call with keyword arguments is directly lowered to construction of a named tuple:

```julia
julia> Meta.@lower f_kwarg(a=1, b=2.0, c=:three)
:($(Expr(:thunk, CodeInfo(
    @ none within `top-level scope'
1 ─ %1 = Core.tuple(:a, :b, :c)
│ %2 = Core.apply_type(Core.NamedTuple, %1)
│ %3 = Core.tuple(1, 2.0, :three)
│ %4 = (%2)(%3)
│ %5 = Core.kwfunc(f_kwarg)
│ %6 = (%5)(%4, f_kwarg)
└── return %6
))))

```

(See also: [Julia Functions · The Julia Language](https://docs.julialang.org/en/v1/devdocs/functions/#Keyword-arguments-1))

So maybe that’s why it’s easier for the compiler to handle this?

---

<div class="post-metadata">

**Author:** ![carstenbauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carstenbauer/32/4981_2.png) [@carstenbauer](https://discourse.julialang.org/u/carstenbauer)\
**Post date:** [February 18, 2020, 8:31am UTC](https://discourse.julialang.org/t/force-specialization-on-kwargs/34789/10 "2020-02-18T08:31:24Z")

</div>

> [@dilumaluthge](#):
>
> If I understand correctly, specialization on keyword arguments would not require keyword arguments to participate in dispatch.

Could you elaborate this? If you have multiple specializations of a method for different combinations of kwarg types how would you utilize, i.e. call, these different specializations if not through dispatch?

---

<div class="post-metadata">

**Author:** ![pablosanjose](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pablosanjose/32/7006_2.png) [@pablosanjose](https://discourse.julialang.org/u/pablosanjose)\
**Post date:** [February 18, 2020, 11:54am UTC](https://discourse.julialang.org/t/force-specialization-on-kwargs/34789/11 "2020-02-18T11:54:38Z")

</div>

I recall reading a post by Jeff, Stefan, or some other core dev saying that code _is_ specialized on kwarg types, just not dispatched on them. I cannot remember where it was, sorry.

---

<div class="post-metadata">

**Author:** ![pablosanjose](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pablosanjose/32/7006_2.png) [@pablosanjose](https://discourse.julialang.org/u/pablosanjose)\
**Post date:** [February 18, 2020, 11:56am UTC](https://discourse.julialang.org/t/force-specialization-on-kwargs/34789/12 "2020-02-18T11:56:43Z")

</div>

Example:

```julia
julia> f(;kw) = values(kw)
f (generic function with 1 method)

julia> @code_llvm f(;kw = 1.0)

; @ REPL[1]:1 within `f##kw'
define double @"julia_f##kw_17248"([1 x double] addrspace(11)* nocapture nonnull readonly dereferenceable(8)) {
top:
; ┌ @ namedtuple.jl:95 within `getindex'
   %1 = getelementptr inbounds [1 x double], [1 x double] addrspace(11)* %0, i64 0, i64 0
; └
  %2 = load double, double addrspace(11)* %1, align 8
  ret double %2
}

julia> @code_llvm f(;kw = 1)

; @ REPL[1]:1 within `f##kw'
define i64 @"julia_f##kw_17215"([1 x i64] addrspace(11)* nocapture nonnull readonly dereferenceable(8)) {
top:
; ┌ @ namedtuple.jl:95 within `getindex'
   %1 = getelementptr inbounds [1 x i64], [1 x i64] addrspace(11)* %0, i64 0, i64 0
; └
  %2 = load i64, i64 addrspace(11)* %1, align 8
  ret i64 %2
}

```

---

<div class="post-metadata">

**Author:** ![oxinabox](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oxinabox/32/206603_2.png) [@oxinabox](https://discourse.julialang.org/u/oxinabox)\
**Post date:** [February 18, 2020, 12:08pm UTC](https://discourse.julialang.org/t/force-specialization-on-kwargs/34789/13 "2020-02-18T12:08:08Z")

</div>

This is my recollection also.  
kwargs _are_ specialized on.  
I believe it was a 1.0 change.  
But I can’t find it in the release notes.

---

<div class="post-metadata">

**Author:** ![carstenbauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carstenbauer/32/4981_2.png) [@carstenbauer](https://discourse.julialang.org/u/carstenbauer)\
**Post date:** [February 18, 2020, 12:33pm UTC](https://discourse.julialang.org/t/force-specialization-on-kwargs/34789/14 "2020-02-18T12:33:41Z")

</div>

Thanks for the example. I understand that kwargs are specialized on. But how is this example different from actually having two methods and dispatching to either one based on the kwargs types?

Otherwise put, what is the difference between methods (in the dispatch sense), which we have one of in your example, and specializations, that is different natively compiled codes for a function (at least two in your example).

Isn’t the fact that different specializations get called depending on the type of your kwargs just a form of dispatch?

Please enlighten me 🙂

---

<div class="post-metadata">

**Author:** ![pablosanjose](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pablosanjose/32/7006_2.png) [@pablosanjose](https://discourse.julialang.org/u/pablosanjose)\
**Post date:** [February 18, 2020, 1:23pm UTC](https://discourse.julialang.org/t/force-specialization-on-kwargs/34789/15 "2020-02-18T13:23:56Z")

</div>

I couldn’t really say, but I suspect dispatch is much more complex than specialization. You can constrain the signature of a function as a whole in many ways that are not just the product of the types of each argument. For example `f(x::T, y::T) where {T}` establishes a correlation of argument types. I suspect the only reason for not allowing multiple dispatch on kwargs is due to this added complexity. If we just had simple dispatch like `f(x::Int, y::Float64)` it would indeed be quite the same as specialization, as far as I understand.

---

<div class="post-metadata">

**Author:** ![carstenbauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carstenbauer/32/4981_2.png) [@carstenbauer](https://discourse.julialang.org/u/carstenbauer)\
**Post date:** [February 24, 2020, 8:42am UTC](https://discourse.julialang.org/t/force-specialization-on-kwargs/34789/16 "2020-02-24T08:42:53Z")

</div>

I guess the major difference is that dispatch, through methods, allows different semantics whereas specialization optimizes a single function body (the meaning is fixed).

---

<div class="post-metadata">

**Author:** ![calebwin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/calebwin/32/14432_2.png) [@calebwin](https://discourse.julialang.org/u/calebwin)\
**Post date:** [March 7, 2022, 1:46pm UTC](https://discourse.julialang.org/t/force-specialization-on-kwargs/34789/17 "2022-03-07T13:46:53Z")

</div>

From reading this thread, it seems like specialization does occur on keyword arguments but no dynamic dispatch (which I don’t totally understand because in my mental model that makes the specialization seem pointless). Is there a way to use `@nospecialize` on keyword arguments?