Cannot use parametric type within function body

Hello, guys!

I’m very confused about parametric typing.

I’ve reduced my problem to a minimal working example:

using LinearAlgebra

function alt(x::Vector{Symmetric{T}}) where {T <: Real}
    return T
end

alt(Vector{Symmetric{Float64}}(undef, 1))

When I try to run this function, it errors out saying that T is undefined!

However, any of these other alternatives work:

function alt(x::Symmetric{T}) where {T <: Real}
    return T
end

function alt(x::Union{Symmetric{T}, Nothing}) where {T <: Real}
    return T
end

And the funny thing is that the parametric type definition works, because the method is correctly dispatched; I can even Juno.@enter into it and run the function up to the line that tries to access the parametric type.

What am I missing here? :frowning:

EDIT: Just adding here that this was tested with Julia 1.2.0 and Julia 1.1.1.

1 Like

To be a full mwe you should add using LinearAlgebra and define a test Vector of Symmetric matrix, so we don’t all have to do that.

But without testing it, you probably need another <:

function alt(x::Vector{<:Symmetric{T}}) where {T <: Real}
    return T
end
1 Like

Thanks for answering!

I already edited the original post.

Your suggestion however, didn’t seem to work:

julia> using LinearAlgebra

julia> function alt(x::Vector{<: Symmetric{T}}) where {T <: Real}
           return T
       end
alt (generic function with 1 method)

julia> alt(Vector{Symmetric{Float64}}(undef, 1))
ERROR: UndefVarError: T not defined
Stacktrace:
 [1] alt(::Array{Symmetric{Float64,S} where S<:(AbstractArray{#s617,2} where #s617<:Float64),1}) at ./REPL[2]:2
 [2] top-level scope at REPL[3]:1

I would like to note that the function I wrote works and is correctly dispatched: the only problem occurs upon using the parametric type T within the body.

Try it with an actual vector of Symmetric :slight_smile:

1 Like

Thank you!

It actually works, but that’s not so good as it only means I don’t have a minimal working example :frowning:
I was making my function types parametric, and had this problem with the following function:

My actual function is this

function smoother(F::Matrix{RT},
                  G::Matrix{RT},
                  a::Vector{Vector{RT}},
                  R::Vector{Symmetric{RT}},
                  m::Vector{Vector{RT}},
                  C::Vector{Symmetric{RT}}) where {RT <: Real}

    n, p = dlm_dimension(F, G)
    T = size(R, 1)

    s = Vector{Vector{RT}}(undef, T)
    S = Vector{Symmetric{RT}}(undef, T)

    s[T] = m[T]
    S[T] = C[T]

    for t = T-1:-1:1
        B = C[t] * G' * inv(R[t+1])
        s[t] = m[t] + B * (s[t+1] - a[t+1])
        S[t] = C[t] - Symmetric(B * (R[t+1] - S[t+1]) * B')
    end

    return s, S
end

As it is, it doesn’t work because it complains RT is undefined.
If I change the Vector{Symmetric{RT}} into Vector{Symmetric{Float64}} the problem disappears. I am in fact calling it with filled up parameters (not undef as the previous mwe).

I can’t come up with a MWE :frowning:

I think the fact that it’s possible for T to be undefined even though dispatch selects the method stems from https://github.com/JuliaLang/julia/pull/23117, but I don’t quite understand why T should be undefined in this case.

But what’s the reason that it shouldn’t work for Vector{Symmetric{Float64}}(undef, 1), or e.g. Symmetric{Float64}[Symmetric(rand(3, 3))], a perfectly valid use case?

Hey @tkoolen, an interesting thing for me is that in my actual example (see post above yours), even though RT is undefined at runtime, I cannot 'compile a function that has the line

RT = typeof(F[1])

because it complains of conflict with the type parameter :confused:

Although this appears to be an interesting corner case (at least to me), in your production code you should probably stop doing stuff like

S = Vector{Symmetric{RT}}(undef, T)

for performance reasons. Note that something like Vector{Symmetric{Float64}} is not a concrete type (it’s a UnionAll); Symmetric has a second type parameter. As a result, you’re incurring a performance cost due to dynamic dispatch any time you touch S. Additionally, your function only accepts stuff like Vector{Symmetric{Float64}}(undef, 1) (a Vector with non-concrete element type), not e.g. [Symmetric(rand(3, 3))] (which does have concrete element type). So you’re inconveniencing callers of the function by forcing them to use a suboptimal storage type, which can be fixed by doing what @Raf proposed in Cannot use parametric type within function body. Once that’s done, you could just use S = similar(C, T) to simultaneously clean up the code and fix the non-concrete element type issue for S.

2 Likes

I think this is the same as: Dispatching on the result of unwrap_unionall seems weird? (which gives an even more MWE).

I’m no type-system expert, but to me it still kind of seems like a bug, e.g. in OPs first example, how can T simultaneously be undefined but also have been verified to be T<:Real ?

1 Like

Thank you a ton!

Great performance tip and now I don’t have the problem of needing to access the type parameter manually anyway!

Though I am still confused about why those original examples didn’t work.

I agree, or at least I don’t understand why this would have to be the case. Not sure if your earlier unwrap_unionall post is exactly related though. Another related paradoxical corner case is https://github.com/JuliaLang/julia/issues/29788, but it doesn’t seem the case in the OP is an exact duplicate of that either. Specifically, in this case there does appear to be a unique correct value for T.

1 Like

It works when Symmetric doesn’t have missing parameters:

julia> alt(Vector{Symmetric{Float64,AbstractArray{Float64,2}}}(undef, 1))
Float64

julia> Vector{Symmetric{Float64}}(undef, 1)
1-element Array{Symmetric{Float64,S} where S<:(AbstractArray{#s623,2} where #s623<:Float64),
1}:
 #undef

julia> Vector{Symmetric{Float64,AbstractArray{Float64,2}}}(undef, 1)
1-element Array{Symmetric{Float64,AbstractArray{Float64,2}},1}:
 #undef

I’m not sure why they don’t both work, I agree it seems like a bug. There is a correct, know value for T here and S being parametric shouldn’t affect that.

Edit: the original example doesn’t work because you need the <: subtype operator when the contained type is parametric, as you are not matching any specific types, but some Symmetric{T,S} from all the possible Symmetric{T,S}. Or something like that.

1 Like

After some private help from @tkoolen and as pointed out by @Raf in the post above, the problem lied with the fact that Symmetric has two parameters.

Upon using Symmetric{RT, Matrix{RT}} everywhere I was previously using just the Symmetric{RT}, not only did the type instabilities (shown by @code_warntype in my previous function) go away but this change allowed me to access the value of RT within the function body.

This means that the instability on the second type from Symmetric was not allowing the access on the parametric type RT.

This was quite a ride and I want to thank everyone for being so helpful!