Using parametric type inside outer parametric constructor

The relevant section of the manual is this one. As @lmiq pointed out, parametric constructors are kinda special, they are the only kinda of function (or function-like?) that can be passed explicit types (without them being a positional argument).

As @lmiq also pointed out, you can put type restrictions directly in the struct definition (this is covered by the link to the manual I provided, and the manual explains how this is equivalent to automatically defining a constructor with a where clause).

Carefully reading this section I noticed one thing. No example in the whole section involves defining an outer constructor (i.e., a constructor outside the struct scope) that had the {T, ...} clause between the function name and the opening parenthesis. The only examples of constructor definitions with this syntax are of inner constructors.

However, as was pointed out in this thread, there is code that defines outer constructors that receive explicit type parameters and work anyway. I agree with @CameronBieganek we can only guess is that there is some kind of ambiguity problem that arises in some situations but not in others.

I would bet on the following: on the minimal example provided by @lmiq the constructor has no way to guess what is the type T. Foo{T}(::UndefInitializer) calls Foo(Vector{T}(undef,2)), what Foo (with no type parameters) will guess to be T? It received a Vector{T}, the struct has only a field x :: S but it is parametrized as Foo{T,S <: Vector{<:T}}, the Vector{T} allow us to assert that S == Vector{T}, but we cannot guess what T (the struct parameter) is, because if it is a Vector{Int} that is being passed (i.e., S == Vector{Int}), we have that Int <: T and, therefore, T can be many things, like Integer or Number. The problem does not happen if T is fixed (instead of being any supertype) because then it is clear that T can only be Int.

2 Likes

I think our terminology is different. When I said running a function, I meant calling a function. Where you said running a function, I said defining a function. :slightly_smiling_face:

This behavior is necessary so that you can define your functions in any order, and so that you can define mutually recursive functions. For example, you can define functions “out of order” like this:

julia> foo() = bar()
foo (generic function with 1 method)

julia> foo()
ERROR: UndefVarError: bar not defined

julia> bar() = 1
bar (generic function with 1 method)

julia> foo()
1

The order doesn’t matter because Julia doesn’t check whether internal variables exist yet when you first define a function.

1 Like

Yeah, I mixed up some words there! Thanks for pointing that out :laughing: :sweat_smile:.

No worries. I should probably stick to “call” rather than “run”, since it’s more precise.

1 Like

Excellent explanation! Now I get it!

I basically disconnected T and S when I defined Foo, and now my constructor requires T to be retrieved from S. That was hard to get my head around that :sweat_smile:

Good recapitulation, I was worried that it was not intelligible because T is used both in the struct parameter and in the constructors with different meaning (on that MWE).

However, this is only relevant for the MWE, because for me, with Julia 1.5.3, your original example works without any changes except removing <: AbstractArray{T, 4} from the struct (because your inheritance makes a specialized Base.show trigger and it expects that you had defined a Base.size for your type to be able to print it). So I think your original code does not make the same mistake intentionally created by @lmiq to trigger the same error message. Julia 1.4.1 probably had a bug that triggered the same error on situations where it did not apply.

1 Like

Uhm… Well, this is even minimal:

julia> struct Foo{T,S<:T} end

julia> Foo{T}(x) where T = Foo{T,T}()

julia> Foo{Float64}(1)
ERROR: UndefVarError: T not defined

Not here:

               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.5.3 (2020-11-09)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> struct Foo{T<:Real, S<:AbstractMatrix{<:T}} 
               data::S
               n::Int

               function Foo{T, S}(data, n) where {T<:Real, S<:AbstractMatrix{<:T}}
                       Base.require_one_based_indexing(data)
                       new{T, S}(data, n)
               end
       end

julia> function Foo(A::AbstractMatrix{T}, n::Int) where {T<:Real}
               return Foo{T, typeof(A)}(A, n)
       end
Foo

julia> function Foo{T}(::UndefInitializer, n::Int) where {T<:Real}
               d = fld(n * (n + 1), 2)
               A = Matrix{T}(undef, d, d)
               return Foo(A, n)
       end

julia> Foo{Float64}(undef, 2)
ERROR: UndefVarError: T not defined
Stacktrace:
 [1] Foo{Float64,S} where S<:(AbstractArray{var"#s1",2} where var"#s1"<:Float64)(::UndefInitializer, ::Int64) at ./REPL[3]:3
 [2] top-level scope at REPL[4]:1

julia>
2 Likes

Yeah, I also ran the original code on 1.4.0, but with the AbstractArray inheritance removed, and I still got the UndefVarError.

Wow, nice example! That really boils it down.

I have no idea what’s going on anymore. :joy:

1 Like

Slightly more minimal:

julia> struct Foo{T, S <: T} end

julia> Foo{T}() where {T} = T

julia> Foo{Integer}()
ERROR: UndefVarError: T not defined
Stacktrace:
 [1] Foo{Integer,S} where S<:Integer() at ./REPL[2]:1
 [2] top-level scope at REPL[3]:1
2 Likes

You see, the function gets specialized to Foo{Integer,S}, but then T in the function is not defined. The problem is the T inside, which looses its meaning when the method is created:

julia> struct Foo{T,S<:T} end

julia> Foo{T}() where T = 1

julia> Foo{Integer}()
1

julia> Foo{T}() where T = x

julia> Foo{Integer}()
ERROR: UndefVarError: x not defined


1 Like

I’m inclined to think this is a bug. I don’t understand why this works:

julia> struct Bar{T, S} end

julia> Bar{T}() where {T} = T

julia> Bar{Integer}()
Integer

but this doesn’t:

julia> struct Foo{T, S <: T} end

julia> Foo{T}() where {T} = T

julia> Foo{Integer}()
ERROR: UndefVarError: T not defined
Stacktrace:
 [1] Foo{Integer,S} where S<:Integer() at ./REPL[2]:1
 [2] top-level scope at REPL[3]:1
2 Likes

I think @Henrique_Becker gave a nice explanation: in your second example, Foo{Integer, Int} would be allowed, but then there’s no way to know what both T and S means within Foo{T}() where {T} = .... In order words, T and S are “connected” in your first example, but “disconnected” in the second. Or am I missing something?

I think it is a type-inference problem:

julia> @code_typed Foo{Int}()
CodeInfo(
1 ─     return $(Expr(:static_parameter, 1))
) => Type{T} where T<:(Union{Int64, S} where S<:Int64)


Of course T is Int64 if T<:Union{Int64,S} and S<:Int64. (I don’t quite get why there is the T<:Union{Int64,S}, really… is that right?).

1 Like

My mistake, I copied the code and forgot to change the commented lines.

It seems clear to me that what I pointed in your original MWE cannot be exactly same that affected OP’s MWE. In OP’s MWE the problem happens because somehow in the first function called T is not recognized within the body of the first function called.

@schneiderfelipe Than for your kind words, but while I think I got it right for @lmiq MWE, I do not think I solved your original problem.

Now… this is funny:

julia> struct Foo{T<:Real, S<:AbstractMatrix{<:T}}
               data::S
               n::Int
       
               function Foo{T, S}(data, n) where {T<:Real, S<:AbstractMatrix{<:T}}
                   println("Foo{T, S}(data, n)")
                   Base.require_one_based_indexing(data)
                   new{T, S}(data, n)
               end
       end
       
       function Foo(A::AbstractMatrix{T}, n::Int) where {T<:Real}
           println("Foo(A, n)")
           return Foo{T, typeof(A)}(A, n)
       end
       
       # Changed the signature below to take both T and S even if S is not used
       function Foo{T, S}(::UndefInitializer, n::Int) where {T<:Real, S <: AbstractMatrix{<:T}}
           println("Foo{T, S}(undef, n)")
           d = fld(n * (n + 1), 2)
       
           # The next two lines should substitute each other,
           # but the commented one throws an error.
       
           A = Matrix{T}(undef, d, d)  # UndefVarError: T not defined
           # A = zeros(d, d)
       
           return Foo(A, n)
       end

julia> Foo{Float64, Matrix{Float64}}(undef, 2)
Foo{T, S}(undef, n)
Foo(A, n)
Foo{T, S}(data, n)
Foo{Float64,Array{Float64,2}}([0.0 0.0 0.0; 0.0 0.0 0.0; 0.0 0.0 0.0], 2)

Why? Why the hell?

My guess now is that, maybe, just maybe, we are not allowed to define an outer constructor taking less type parameters than the struct itself?

4 Likes

It seems to me that the behavior of the MWE is determined by some code lowering and/or other implementation details that are not specified in the manual. I still feel that the current behavior of the MWE is incorrect, but of course I could be wrong.

We might have to file a Github issue and have the experts weigh in.

3 Likes

or just mark one of them here?

1 Like

I’m not sure I want to tag a bunch of people here… :joy:

1 Like
3 Likes

This feels like a plausible explanation. I vaguely recall that static parameters are only ever assigned by the compiler if they are uniquely identified. The compiler needs to find, if possible, a unique T in order for T to be available in the method body. Somehow the constraint on S introduces a weird edge case and the compiler just gives up. Ironically, it is smart enough to know that

julia> (Union{Int64,S} where {S<:Int64}) == Int64
true

but not smart enough to conclude the static type must be Int64. I think Jeff mentioned this issue in What’s bad about Julia? where types can have extents that just drive the compiler crazy.

3 Likes

Thanks for opening the issue @CameronBieganek!

2 Likes