Matching any vector in function dispatch

New to Julia
1

I’m having trouble with function dispatch where I want to have two different implementations when the values in a dictionary are vectors or not

These work as I would expect:

function fn(val::Vector{Any})
    @info "fn, any vector"
    @show typeof(val)
end

function fn(val::Vector)
    @info "fn, vector"
    @show typeof(val)
end

function fn(val)
    @info "fn, scalar"
    @show typeof(val)
end

fn(1)
# [ Info: fn, scalar
# typeof(val) = Int64

fn(true)
# [ Info: fn, scalar
# typeof(val) = Bool

fn("foo")
# [ Info: fn, scalar
# typeof(val) = String

fn([1, 2])
# [ Info: fn, vector
# typeof(val) = Vector{Int64}

fn([true, false])
# [ Info: fn, vector
# typeof(val) = Vector{Bool}

fn(["foo", "bar"])
# [ Info: fn, vector
# typeof(val) = Vector{String}

fn([1, "foo"])
# [ Info: fn, any vector
# typeof(val) = Vector{Any}

However in the examples below, I don’t understand how the vector versions of the functions are being picked.

function fn(val::Dict{Symbol,Vector{Any}})
    @info "fn, dict of any vector"
    @show typeof(val)
end

function fn(val::Dict{Symbol,Vector})
    @info "fn, dict of vector"
    @show typeof(val)
end

function fn(val::Dict)
    @info "fn, dict of scalar"
    @show typeof(val)
end

## as expected 
fn(Dict(:a => 1, :b => 2))
# [ Info: fn, dict of scalar
# typeof(val) = Dict{Symbol, Int64}

fn(Dict(:a => true, :b => false))
# [ Info: fn, dict of scalar
# typeof(val) = Dict{Symbol, Bool}

fn(Dict(:a => "foo", :b => "bar"))
# [ Info: fn, dict of scalar
# typeof(val) = Dict{Symbol, String}

fn(Dict(:a => [1] , :b => [2]))
# [ Info: fn, dict of scalar    <- expected dict of vector
# typeof(val) = Dict{Symbol, Vector{Int64}}

fn(Dict(:a => [true], :b => [false]))
# [ Info: fn, dict of scalar    <- expected dict of vector
# typeof(val) = Dict{Symbol, Vector{Bool}}

fn(Dict(:a => ["foo"], :b => ["bar"]))
# [ Info: fn, dict of scalar    <- expected dict of vector
# typeof(val) = Dict{Symbol, Vector{String}}

fn(Dict(:a => [1, "foo"], :b => ["bar"]))
# [ Info: fn, dict of vector    <- expected dict of any vector
# typeof(val) = Dict{Symbol, Vector}

Can someone help me understand what’s going on, and recommend how should I be typing this? When the dictionary values are vectors, I want to have slightly different behaviour.

1 Like
2

Subtyping with parametric types is slightly counter-intuitive. Take a look at

4 Likes
3

Great explanation, thanks for sharing the link!

I just wanted to add a link to the relevant section of the docs, as well as a suggestion for solving the original issue:

# Fallback for "dict with symbol as key and anything as values"
fn(val::Dict{Symbol,T}) where T

# "dicts with symbols as keys and values that are a Vector of any kind (not just `Vector{Any}`!)
fn(val::Dict{Symbol,V}) where V <: Vector

# "dicts with symbols as keys and values that are `Vector{Any}` (note that this is more specific than the above)"
fn(val::Dict{Symbol,Vector{Any}})

I’m not sure whether this is what you want though.

The reason why your last example doesn’t work is (in my opinion) also because of a small gotcha:

julia> Dict(:a => [1, "foo"], :b => ["bar"])
Dict{Symbol, Vector} with 2 entries:
  :a => Any[1, "foo"]
  :b => ["bar"]

The resulting dict contains one Vector{Any} and one Vector{String} and Julia decides to specify Vector as the common supertype and Dict parameter. However, there are cases, where the result is (perhaps) the one you might expect:

julia> [[1, "foo"], ["bar"]]
2-element Vector{Vector{Any}}:
 [1, "foo"]
 ["bar"]

Although it looks like the same situation as with the dict above, Julia tries to make the eltype of the vector concrete here and converts the second element to a Vector{Any}. I don’t know why, but I guess it’s for better performance (?).

There is also a small gotcha when it comes to placing the type variables:

julia> Dict{Symbol, Vector{Int}} <: Dict{Symbol, Vector{I}} where I <: Integer
true

julia> Dict{Symbol, Vector{Int}} <: Dict{Symbol, Vector{I} where I <: Integer}
false

The two types on the right-hand side look very similar, but they are different: the first case, the I is a type variable of the whole type and can be any subtype of Integer, so the expression is something like Union{Dict{Symbol, Vector{Int}}, Dict{Symbol, Vector{Int32}}, Dict{Symbol, Vector{UInt32}}, ...}. The type on the left-hand side is contained in this union.

But in the second case, the variable I is “bound” to the second parameter of the Dict and would translate to something like Dict{Symbol, Union{Vector{Int}, Vector{UInt32}, ...}}. Because of type invariance, the type on the left-hand side is not a subtype of this, although of course Vector{Int} <: Vector{I} where I <: Integer

This is why your Dict{Symbol, Vector} is almost the right thing (Vector is essentially Vector{T} where T), but it doesn’t behave the same as Dict{Symbol, Vector{T}} where T where the type variable can apply to the whole dict type.

Another approach might be to somehow split the logic into a separate function call, i.e. have fn(val::Dict{Symbol,T}) where T as a generic function and either manually select which types inside the function with T isa Vector etc. or by writing an “inner function” that takes the type as argument. Then you could use the first function you wrote above which behaves as you would expect. You can wrap the types in a Type (see here for more explanations)

function fn(x::Dict{Symbol, T}) where T
    ...
    inner(T, ...)
    ...
end 

function inner(::Type{T}, ...) where T <: Vector
    ...
end

For nested parametric types this may or may not make it easier to wrap one’s head around the path that multiple dispatch takes.

1 Like
4

Both of you thanks a lot!

@gdalle your notes are amazing, it was invaluable to understand the concept. I was completely misunderstanding how container types work.

@Sevi your comment on the placement of the type variable was the final missing piece of understanding for me. Also thanks for the alternative suggestions. I’ll see what works best for my actual use case.

For completeness, modifying the MWE above like this makes everything work as intended:

function fn(val::Dict{Symbol,Vector{T} where T})
    @info "fn, dict of any vectors (also mixed types)"
    @show typeof(val)
end

function fn(val::Dict{Symbol,Vector{T}}) where T <: Union{Int64, String, Bool}
    @info "fn, dict of vector containing T ∈ union"
    @show typeof(val)
end

function fn(val::Dict{Symbol, T}) where T
    @info "fn, dict of scalar"
    @show typeof(val)
end

fn(Dict(:a => 1, :b => 2))
# [ Info: fn, dict of scalar
# typeof(val) = Dict{Symbol, Int64}

fn(Dict(:a => true, :b => false))
# [ Info: fn, dict of scalar
# typeof(val) = Dict{Symbol, Bool}

fn(Dict(:a => "foo", :b => "bar"))
# [ Info: fn, dict of scalar
# typeof(val) = Dict{Symbol, String}

fn(Dict(:a => [1] , :b => [2]))
# [ Info: fn, dict of vector containing T ∈ union
# typeof(val) = Dict{Symbol, Vector{Int64}}

fn(Dict(:a => [true], :b => [false]))
# [ Info: fn, dict of vector containing T ∈ union
# typeof(val) = Dict{Symbol, Vector{Bool}}

fn(Dict(:a => ["foo"], :b => ["bar"]))
# [ Info: fn, dict of vector containing T ∈ union
# typeof(val) = Dict{Symbol, Vector{String}}

fn(Dict(:a => [1, "foo"], :b => ["bar"]))
# [ Info: fn, dict of any vectors (also mixed types)
# typeof(val) = Dict{Symbol, Vector}
1 Like
5

Not my notes, all credit goes to @lmiq :wink:

3 Likes