Using parametric type inside outer parametric constructor

My best guess is that there is some kind of ambiguity in the various constructors that you’ve defined, but I can’t put my finger on what that would be.

1 Like

Removing the <: from the type definition makes it work:

This works:

julia> struct Foo{T<:Real, S<:AbstractMatrix{T}}
               data::S
               n::Int
       end

julia> Foo(A::AbstractMatrix{T}, n::Int) where {T<:Real} = Foo{T, typeof(A)}(A, n)
Foo

julia> Foo{T}(::UndefInitializer,n) where {T<:Real} = Foo(Matrix{T}(undef,n,n),n)

julia> Foo{Float64}(undef,2)
Foo{Float64,Array{Float64,2}}([0.0 0.0; 0.0 0.0], 2)

julia>

This does not:

julia> struct Foo{T<:Real, S<:AbstractMatrix{<:T}} # the difference is the <: here
               data::S
               n::Int
       end

julia> Foo(A::AbstractMatrix{T}, n::Int) where {T<:Real} = Foo{T, typeof(A)}(A, n)
Foo

julia> Foo{T}(::UndefInitializer,n) where {T<:Real} = Foo(Matrix{T}(undef,n,n),n)

julia> Foo{Float64}(undef,2)
ERROR: UndefVarError: T not defined

Minimal example:

julia> struct Foo{T,S <: Vector{<:T}}
         x :: S
       end

julia> Foo{T}(::UndefInitializer) where T = Foo(Vector{T}(undef,2))

julia> Foo{Float64}(undef)
ERROR: UndefVarError: T not defined

2 Likes

Awesome finding! That is subtle but definitely solves my issue.

But now, two questions:

  1. Do you know why this is so? What constitutes the error?
  2. Is there a fundamental reason why this AbstractMatrix{<:T} idiom is used in the Julia codebase, e.g., in LinearAlgebra? In fact, I wrote the first part of my example based on the definition of Hermitian (see here), and the last part of it looking into outer constructors of Array{T, 2} (already mentioned, link here).

No, I don’t. But those things mean different things.

In one you can do:

julia> struct Foo{T,S <: Vector{<:T}}
         x :: S
       end

julia> Foo{Real,Vector{Float64}}(zeros(2))
Foo{Real,Array{Float64,1}}([0.0, 0.0])

In the other you can’t:

julia> struct Foo2{T,S<:Vector{T}}
         x :: S
       end

julia> Foo2{Real,Vector{Float64}}(zeros(2))
ERROR: TypeError: in Foo2, in S, expected S<:Array{T,1}, got Type{Array{Float64,1}}

Interestingly, in the first case, with <:, you cannot build the struct with mixed-type arrays, and we get the same error as you:

julia> Foo(Union{Float64,Int64}[1,1.0])
ERROR: UndefVarError: T not defined

(Maybe this gives a hint on what is going on)

In the second case, without the <:, you can:

julia> Foo2(Union{Float64,Int64}[1,1.0])
Foo2{Union{Float64, Int64},Array{Union{Float64, Int64},1}}(Union{Float64, Int64}[1, 1.0])


1 Like

Nice! I think I have a better understanding of it now.

But that’s still odd that everything “works” when I omit the A = Matrix{T}(undef, d, d) in the last function of my example (that is, Julia doesn’t complain if T is not mentioned, even though the function signature is the same). Of course, I would rather not data to zeros though…

1 Like

I’m still a bit confused. The suggestion from @lmiq made me wonder about a related question that seems odd to me. I’ve opened a separate thread for that specific question:

1 Like

Thanks, @CameronBieganek! This seems to be an issue as well, but here I’ve mentioned that there should be more than that in the present case.

In fact, by changing a single line in the function body (same signature, same parameters, etc.), everything “works” as expected.

Yes, it has something to do with the undef initialization.

Well, that part I can explain. It’s perfectly possible to define functions that refer to variables that don’t exist. Julia doesn’t throw an error until you run the function and it encounters a variable that has not been defined yet:

julia> function foo()
           local x
           return x
       end
foo (generic function with 1 method)

julia> foo()
ERROR: UndefVarError: x not defined

EDIT: As I commented below, I should probably replace “run the function” with “call the function” in order to clarify the difference between defining a function and calling a function.

1 Like

The point is that, in my example, running the function does not throw an error. In the context of my example, the following throws an error,

function Foo{T}(::UndefInitializer, n::Int) where {T<:Real}
	d = fld(n * (n + 1), 2)
	A = Matrix{T}(undef, d, d)
	return Foo(A, n)
end

Foo{Float64}(undef, 2);
# => UndefVarError: T not defined

but the following does not:

function Foo{T}(::UndefInitializer, n::Int) where {T<:Real}
	d = fld(n * (n + 1), 2)
	A = zeros(d, d)
	return Foo(A, n)
end

Foo{Float64}(undef, 2);
# works fine

The relevant section of the manual is this one. As @lmiq pointed out, parametric constructors are kinda special, they are the only kinda of function (or function-like?) that can be passed explicit types (without them being a positional argument).

As @lmiq also pointed out, you can put type restrictions directly in the struct definition (this is covered by the link to the manual I provided, and the manual explains how this is equivalent to automatically defining a constructor with a where clause).

Carefully reading this section I noticed one thing. No example in the whole section involves defining an outer constructor (i.e., a constructor outside the struct scope) that had the {T, ...} clause between the function name and the opening parenthesis. The only examples of constructor definitions with this syntax are of inner constructors.

However, as was pointed out in this thread, there is code that defines outer constructors that receive explicit type parameters and work anyway. I agree with @CameronBieganek we can only guess is that there is some kind of ambiguity problem that arises in some situations but not in others.

I would bet on the following: on the minimal example provided by @lmiq the constructor has no way to guess what is the type T. Foo{T}(::UndefInitializer) calls Foo(Vector{T}(undef,2)), what Foo (with no type parameters) will guess to be T? It received a Vector{T}, the struct has only a field x :: S but it is parametrized as Foo{T,S <: Vector{<:T}}, the Vector{T} allow us to assert that S == Vector{T}, but we cannot guess what T (the struct parameter) is, because if it is a Vector{Int} that is being passed (i.e., S == Vector{Int}), we have that Int <: T and, therefore, T can be many things, like Integer or Number. The problem does not happen if T is fixed (instead of being any supertype) because then it is clear that T can only be Int.

2 Likes

I think our terminology is different. When I said running a function, I meant calling a function. Where you said running a function, I said defining a function. :slightly_smiling_face:

This behavior is necessary so that you can define your functions in any order, and so that you can define mutually recursive functions. For example, you can define functions “out of order” like this:

julia> foo() = bar()
foo (generic function with 1 method)

julia> foo()
ERROR: UndefVarError: bar not defined

julia> bar() = 1
bar (generic function with 1 method)

julia> foo()
1

The order doesn’t matter because Julia doesn’t check whether internal variables exist yet when you first define a function.

1 Like

Yeah, I mixed up some words there! Thanks for pointing that out :laughing: :sweat_smile:.

No worries. I should probably stick to “call” rather than “run”, since it’s more precise.

1 Like

Excellent explanation! Now I get it!

I basically disconnected T and S when I defined Foo, and now my constructor requires T to be retrieved from S. That was hard to get my head around that :sweat_smile:

Good recapitulation, I was worried that it was not intelligible because T is used both in the struct parameter and in the constructors with different meaning (on that MWE).

However, this is only relevant for the MWE, because for me, with Julia 1.5.3, your original example works without any changes except removing <: AbstractArray{T, 4} from the struct (because your inheritance makes a specialized Base.show trigger and it expects that you had defined a Base.size for your type to be able to print it). So I think your original code does not make the same mistake intentionally created by @lmiq to trigger the same error message. Julia 1.4.1 probably had a bug that triggered the same error on situations where it did not apply.

1 Like

Uhm… Well, this is even minimal:

julia> struct Foo{T,S<:T} end

julia> Foo{T}(x) where T = Foo{T,T}()

julia> Foo{Float64}(1)
ERROR: UndefVarError: T not defined

Not here:

               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.5.3 (2020-11-09)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> struct Foo{T<:Real, S<:AbstractMatrix{<:T}} 
               data::S
               n::Int

               function Foo{T, S}(data, n) where {T<:Real, S<:AbstractMatrix{<:T}}
                       Base.require_one_based_indexing(data)
                       new{T, S}(data, n)
               end
       end

julia> function Foo(A::AbstractMatrix{T}, n::Int) where {T<:Real}
               return Foo{T, typeof(A)}(A, n)
       end
Foo

julia> function Foo{T}(::UndefInitializer, n::Int) where {T<:Real}
               d = fld(n * (n + 1), 2)
               A = Matrix{T}(undef, d, d)
               return Foo(A, n)
       end

julia> Foo{Float64}(undef, 2)
ERROR: UndefVarError: T not defined
Stacktrace:
 [1] Foo{Float64,S} where S<:(AbstractArray{var"#s1",2} where var"#s1"<:Float64)(::UndefInitializer, ::Int64) at ./REPL[3]:3
 [2] top-level scope at REPL[4]:1

julia>
2 Likes

Yeah, I also ran the original code on 1.4.0, but with the AbstractArray inheritance removed, and I still got the UndefVarError.

Wow, nice example! That really boils it down.

I have no idea what’s going on anymore. :joy:

1 Like

Slightly more minimal:

julia> struct Foo{T, S <: T} end

julia> Foo{T}() where {T} = T

julia> Foo{Integer}()
ERROR: UndefVarError: T not defined
Stacktrace:
 [1] Foo{Integer,S} where S<:Integer() at ./REPL[2]:1
 [2] top-level scope at REPL[3]:1
2 Likes

You see, the function gets specialized to Foo{Integer,S}, but then T in the function is not defined. The problem is the T inside, which looses its meaning when the method is created:

julia> struct Foo{T,S<:T} end

julia> Foo{T}() where T = 1

julia> Foo{Integer}()
1

julia> Foo{T}() where T = x

julia> Foo{Integer}()
ERROR: UndefVarError: x not defined


1 Like