Using parametric type inside outer parametric constructor

I’m still a bit confused. The suggestion from @lmiq made me wonder about a related question that seems odd to me. I’ve opened a separate thread for that specific question:

1 Like

Thanks, @CameronBieganek! This seems to be an issue as well, but here I’ve mentioned that there should be more than that in the present case.

In fact, by changing a single line in the function body (same signature, same parameters, etc.), everything “works” as expected.

Yes, it has something to do with the undef initialization.

Well, that part I can explain. It’s perfectly possible to define functions that refer to variables that don’t exist. Julia doesn’t throw an error until you run the function and it encounters a variable that has not been defined yet:

julia> function foo()
           local x
           return x
       end
foo (generic function with 1 method)

julia> foo()
ERROR: UndefVarError: x not defined

EDIT: As I commented below, I should probably replace “run the function” with “call the function” in order to clarify the difference between defining a function and calling a function.

1 Like

The point is that, in my example, running the function does not throw an error. In the context of my example, the following throws an error,

function Foo{T}(::UndefInitializer, n::Int) where {T<:Real}
	d = fld(n * (n + 1), 2)
	A = Matrix{T}(undef, d, d)
	return Foo(A, n)
end

Foo{Float64}(undef, 2);
# => UndefVarError: T not defined

but the following does not:

function Foo{T}(::UndefInitializer, n::Int) where {T<:Real}
	d = fld(n * (n + 1), 2)
	A = zeros(d, d)
	return Foo(A, n)
end

Foo{Float64}(undef, 2);
# works fine

The relevant section of the manual is this one. As @lmiq pointed out, parametric constructors are kinda special, they are the only kinda of function (or function-like?) that can be passed explicit types (without them being a positional argument).

As @lmiq also pointed out, you can put type restrictions directly in the struct definition (this is covered by the link to the manual I provided, and the manual explains how this is equivalent to automatically defining a constructor with a where clause).

Carefully reading this section I noticed one thing. No example in the whole section involves defining an outer constructor (i.e., a constructor outside the struct scope) that had the {T, ...} clause between the function name and the opening parenthesis. The only examples of constructor definitions with this syntax are of inner constructors.

However, as was pointed out in this thread, there is code that defines outer constructors that receive explicit type parameters and work anyway. I agree with @CameronBieganek we can only guess is that there is some kind of ambiguity problem that arises in some situations but not in others.

I would bet on the following: on the minimal example provided by @lmiq the constructor has no way to guess what is the type T. Foo{T}(::UndefInitializer) calls Foo(Vector{T}(undef,2)), what Foo (with no type parameters) will guess to be T? It received a Vector{T}, the struct has only a field x :: S but it is parametrized as Foo{T,S <: Vector{<:T}}, the Vector{T} allow us to assert that S == Vector{T}, but we cannot guess what T (the struct parameter) is, because if it is a Vector{Int} that is being passed (i.e., S == Vector{Int}), we have that Int <: T and, therefore, T can be many things, like Integer or Number. The problem does not happen if T is fixed (instead of being any supertype) because then it is clear that T can only be Int.

2 Likes

I think our terminology is different. When I said running a function, I meant calling a function. Where you said running a function, I said defining a function. :slightly_smiling_face:

This behavior is necessary so that you can define your functions in any order, and so that you can define mutually recursive functions. For example, you can define functions “out of order” like this:

julia> foo() = bar()
foo (generic function with 1 method)

julia> foo()
ERROR: UndefVarError: bar not defined

julia> bar() = 1
bar (generic function with 1 method)

julia> foo()
1

The order doesn’t matter because Julia doesn’t check whether internal variables exist yet when you first define a function.

1 Like

Yeah, I mixed up some words there! Thanks for pointing that out :laughing: :sweat_smile:.

No worries. I should probably stick to “call” rather than “run”, since it’s more precise.

1 Like

Excellent explanation! Now I get it!

I basically disconnected T and S when I defined Foo, and now my constructor requires T to be retrieved from S. That was hard to get my head around that :sweat_smile:

Good recapitulation, I was worried that it was not intelligible because T is used both in the struct parameter and in the constructors with different meaning (on that MWE).

However, this is only relevant for the MWE, because for me, with Julia 1.5.3, your original example works without any changes except removing <: AbstractArray{T, 4} from the struct (because your inheritance makes a specialized Base.show trigger and it expects that you had defined a Base.size for your type to be able to print it). So I think your original code does not make the same mistake intentionally created by @lmiq to trigger the same error message. Julia 1.4.1 probably had a bug that triggered the same error on situations where it did not apply.

1 Like

Uhm… Well, this is even minimal:

julia> struct Foo{T,S<:T} end

julia> Foo{T}(x) where T = Foo{T,T}()

julia> Foo{Float64}(1)
ERROR: UndefVarError: T not defined

Not here:

               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.5.3 (2020-11-09)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> struct Foo{T<:Real, S<:AbstractMatrix{<:T}} 
               data::S
               n::Int

               function Foo{T, S}(data, n) where {T<:Real, S<:AbstractMatrix{<:T}}
                       Base.require_one_based_indexing(data)
                       new{T, S}(data, n)
               end
       end

julia> function Foo(A::AbstractMatrix{T}, n::Int) where {T<:Real}
               return Foo{T, typeof(A)}(A, n)
       end
Foo

julia> function Foo{T}(::UndefInitializer, n::Int) where {T<:Real}
               d = fld(n * (n + 1), 2)
               A = Matrix{T}(undef, d, d)
               return Foo(A, n)
       end

julia> Foo{Float64}(undef, 2)
ERROR: UndefVarError: T not defined
Stacktrace:
 [1] Foo{Float64,S} where S<:(AbstractArray{var"#s1",2} where var"#s1"<:Float64)(::UndefInitializer, ::Int64) at ./REPL[3]:3
 [2] top-level scope at REPL[4]:1

julia>
2 Likes

Yeah, I also ran the original code on 1.4.0, but with the AbstractArray inheritance removed, and I still got the UndefVarError.

Wow, nice example! That really boils it down.

I have no idea what’s going on anymore. :joy:

1 Like

Slightly more minimal:

julia> struct Foo{T, S <: T} end

julia> Foo{T}() where {T} = T

julia> Foo{Integer}()
ERROR: UndefVarError: T not defined
Stacktrace:
 [1] Foo{Integer,S} where S<:Integer() at ./REPL[2]:1
 [2] top-level scope at REPL[3]:1
2 Likes

You see, the function gets specialized to Foo{Integer,S}, but then T in the function is not defined. The problem is the T inside, which looses its meaning when the method is created:

julia> struct Foo{T,S<:T} end

julia> Foo{T}() where T = 1

julia> Foo{Integer}()
1

julia> Foo{T}() where T = x

julia> Foo{Integer}()
ERROR: UndefVarError: x not defined


1 Like

I’m inclined to think this is a bug. I don’t understand why this works:

julia> struct Bar{T, S} end

julia> Bar{T}() where {T} = T

julia> Bar{Integer}()
Integer

but this doesn’t:

julia> struct Foo{T, S <: T} end

julia> Foo{T}() where {T} = T

julia> Foo{Integer}()
ERROR: UndefVarError: T not defined
Stacktrace:
 [1] Foo{Integer,S} where S<:Integer() at ./REPL[2]:1
 [2] top-level scope at REPL[3]:1
2 Likes

I think @Henrique_Becker gave a nice explanation: in your second example, Foo{Integer, Int} would be allowed, but then there’s no way to know what both T and S means within Foo{T}() where {T} = .... In order words, T and S are “connected” in your first example, but “disconnected” in the second. Or am I missing something?

I think it is a type-inference problem:

julia> @code_typed Foo{Int}()
CodeInfo(
1 ─     return $(Expr(:static_parameter, 1))
) => Type{T} where T<:(Union{Int64, S} where S<:Int64)


Of course T is Int64 if T<:Union{Int64,S} and S<:Int64. (I don’t quite get why there is the T<:Union{Int64,S}, really… is that right?).

1 Like

My mistake, I copied the code and forgot to change the commented lines.

It seems clear to me that what I pointed in your original MWE cannot be exactly same that affected OP’s MWE. In OP’s MWE the problem happens because somehow in the first function called T is not recognized within the body of the first function called.

@schneiderfelipe Than for your kind words, but while I think I got it right for @lmiq MWE, I do not think I solved your original problem.

Now… this is funny:

julia> struct Foo{T<:Real, S<:AbstractMatrix{<:T}}
               data::S
               n::Int
       
               function Foo{T, S}(data, n) where {T<:Real, S<:AbstractMatrix{<:T}}
                   println("Foo{T, S}(data, n)")
                   Base.require_one_based_indexing(data)
                   new{T, S}(data, n)
               end
       end
       
       function Foo(A::AbstractMatrix{T}, n::Int) where {T<:Real}
           println("Foo(A, n)")
           return Foo{T, typeof(A)}(A, n)
       end
       
       # Changed the signature below to take both T and S even if S is not used
       function Foo{T, S}(::UndefInitializer, n::Int) where {T<:Real, S <: AbstractMatrix{<:T}}
           println("Foo{T, S}(undef, n)")
           d = fld(n * (n + 1), 2)
       
           # The next two lines should substitute each other,
           # but the commented one throws an error.
       
           A = Matrix{T}(undef, d, d)  # UndefVarError: T not defined
           # A = zeros(d, d)
       
           return Foo(A, n)
       end

julia> Foo{Float64, Matrix{Float64}}(undef, 2)
Foo{T, S}(undef, n)
Foo(A, n)
Foo{T, S}(data, n)
Foo{Float64,Array{Float64,2}}([0.0 0.0 0.0; 0.0 0.0 0.0; 0.0 0.0 0.0], 2)

Why? Why the hell?

My guess now is that, maybe, just maybe, we are not allowed to define an outer constructor taking less type parameters than the struct itself?

4 Likes

It seems to me that the behavior of the MWE is determined by some code lowering and/or other implementation details that are not specified in the manual. I still feel that the current behavior of the MWE is incorrect, but of course I could be wrong.

We might have to file a Github issue and have the experts weigh in.

3 Likes