Using parametric type inside outer parametric constructor

General Usage
, ,
1

Hi,

I can’t figure out how to use a parametric type inside a constructor. Namely, the following single commented line can’t be used:

struct Foo{T<:Real, S<:AbstractMatrix{<:T}} <: AbstractArray{T, 4}
	data::S
	n::Int

	function Foo{T, S}(data, n) where {T<:Real, S<:AbstractMatrix{<:T}}
		Base.require_one_based_indexing(data)
		new{T, S}(data, n)
	end
end

function Foo(A::AbstractMatrix{T}, n::Int) where {T<:Real}
	return Foo{T, typeof(A)}(A, n)
end

function Foo{T}(::UndefInitializer, n::Int) where {T<:Real}
	d = fld(n * (n + 1), 2)

    # The next two lines should substitute each other,
    # but the commented one throws an error.

	# A = Matrix{T}(undef, d, d)  # UndefVarError: T not defined
	A = zeros(d, d)

	return Foo(A, n)
end

The function Foo{T}(::UndefInitializer, n::Int) where {T<:Real} works as long as T is not mentioned anywhere in its body. Otherwise, a UndefVarError: T not defined is thrown:

# ; is needed because I'm omitting the definitions of size and
# getindex, which are complicated and required by the REPL to
# show AbstractArrays

julia> Foo{Float64}(undef, 2);
ERROR: UndefVarError: T not defined
Stacktrace:
 [1] Foo{Float64,S} where S<:(AbstractArray{#s1,2} where #s1<:Float64)(::UndefInitializer, ::Int64) at ./REPL[5]:7
 [2] top-level scope at REPL[6]:1

Version information:

julia> versioninfo()
Julia Version 1.4.1
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: Intel(R) Core(TM) i7-5500U CPU @ 2.40GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-8.0.1 (ORCJIT, broadwell)
Environment:
  JULIA_NUM_THREADS = 4

What am I doing wrong?

2

Perhaps the easiest is to do:

julia> struct Foo{T}
         x :: Vector{T}
       end

julia> Foo(T::DataType,n) = Foo(Vector{T}(undef,n))
Foo

julia> Foo(Float64,3)
Foo{Float64}([0.0, 0.0, 0.0])

Understanding the error I think is more complicated (at least for me). But part of the problem is that functions are not parametric. That is, you cannot write something like:

julia> f{T}(x) = T(x)
ERROR: UndefVarError: f not defined

if f is not previously a data type.

Thus, when you are defining Foo{T}(x) as a constructor, you are actually writting a function like object, which instantiates Foo with type T, but T is not a parameter of that function, such that it is not defined inside it. Something like that.

2 Likes
3

Thanks that is an excellent answer.

What still bothers me is that there are examples in the Julia codebase that work as I wished my code would! See, for example (from @edit Array{Float64}(undef, 2), see file here),

...
# type but not dimensionality specified
Array{T}(::UndefInitializer, m::Int) where {T} = Array{T,1}(undef, m)
...

How come this works there but not here? How is that different from my example?

Two similarity points that I would like to emphasize:

  • Both are definitions of a parametric constructor that require a single parameter, but both types have two.
  • Both definitions are like TypeName{T}(...no T here...) where {T} = ...
4

Yes, that is very strange. As you said, this sort of thing normally works just fine. Here’s a simpler example that works as expected:

struct A{T}
    x::T
end

function A{T}() where T
    A(zero(T))
end

julia> A{Float64}()
A{Float64}(0.0)

I don’t want to jump to conclusions, but this almost seems like a bug in Julia. :thinking:

2 Likes
5

My best guess is that there is some kind of ambiguity in the various constructors that you’ve defined, but I can’t put my finger on what that would be.

1 Like
6

Removing the <: from the type definition makes it work:

This works:

julia> struct Foo{T<:Real, S<:AbstractMatrix{T}}
               data::S
               n::Int
       end

julia> Foo(A::AbstractMatrix{T}, n::Int) where {T<:Real} = Foo{T, typeof(A)}(A, n)
Foo

julia> Foo{T}(::UndefInitializer,n) where {T<:Real} = Foo(Matrix{T}(undef,n,n),n)

julia> Foo{Float64}(undef,2)
Foo{Float64,Array{Float64,2}}([0.0 0.0; 0.0 0.0], 2)

julia>

This does not:

julia> struct Foo{T<:Real, S<:AbstractMatrix{<:T}} # the difference is the <: here
               data::S
               n::Int
       end

julia> Foo(A::AbstractMatrix{T}, n::Int) where {T<:Real} = Foo{T, typeof(A)}(A, n)
Foo

julia> Foo{T}(::UndefInitializer,n) where {T<:Real} = Foo(Matrix{T}(undef,n,n),n)

julia> Foo{Float64}(undef,2)
ERROR: UndefVarError: T not defined

Minimal example:

julia> struct Foo{T,S <: Vector{<:T}}
         x :: S
       end

julia> Foo{T}(::UndefInitializer) where T = Foo(Vector{T}(undef,2))

julia> Foo{Float64}(undef)
ERROR: UndefVarError: T not defined
2 Likes
7

Awesome finding! That is subtle but definitely solves my issue.

But now, two questions:

  1. Do you know why this is so? What constitutes the error?
  2. Is there a fundamental reason why this AbstractMatrix{<:T} idiom is used in the Julia codebase, e.g., in LinearAlgebra? In fact, I wrote the first part of my example based on the definition of Hermitian (see here), and the last part of it looking into outer constructors of Array{T, 2} (already mentioned, link here).
8

No, I don’t. But those things mean different things.

In one you can do:

julia> struct Foo{T,S <: Vector{<:T}}
         x :: S
       end

julia> Foo{Real,Vector{Float64}}(zeros(2))
Foo{Real,Array{Float64,1}}([0.0, 0.0])

In the other you can’t:

julia> struct Foo2{T,S<:Vector{T}}
         x :: S
       end

julia> Foo2{Real,Vector{Float64}}(zeros(2))
ERROR: TypeError: in Foo2, in S, expected S<:Array{T,1}, got Type{Array{Float64,1}}

Interestingly, in the first case, with <:, you cannot build the struct with mixed-type arrays, and we get the same error as you:

julia> Foo(Union{Float64,Int64}[1,1.0])
ERROR: UndefVarError: T not defined

(Maybe this gives a hint on what is going on)

In the second case, without the <:, you can:

julia> Foo2(Union{Float64,Int64}[1,1.0])
Foo2{Union{Float64, Int64},Array{Union{Float64, Int64},1}}(Union{Float64, Int64}[1, 1.0])
1 Like
9

Nice! I think I have a better understanding of it now.

But that’s still odd that everything “works” when I omit the A = Matrix{T}(undef, d, d) in the last function of my example (that is, Julia doesn’t complain if T is not mentioned, even though the function signature is the same). Of course, I would rather not data to zeros though…

1 Like
10

I’m still a bit confused. The suggestion from @lmiq made me wonder about a related question that seems odd to me. I’ve opened a separate thread for that specific question:

1 Like
11

Thanks, @CameronBieganek! This seems to be an issue as well, but here I’ve mentioned that there should be more than that in the present case.

In fact, by changing a single line in the function body (same signature, same parameters, etc.), everything “works” as expected.

12

Yes, it has something to do with the undef initialization.

13

Well, that part I can explain. It’s perfectly possible to define functions that refer to variables that don’t exist. Julia doesn’t throw an error until you run the function and it encounters a variable that has not been defined yet:

julia> function foo()
           local x
           return x
       end
foo (generic function with 1 method)

julia> foo()
ERROR: UndefVarError: x not defined

EDIT: As I commented below, I should probably replace “run the function” with “call the function” in order to clarify the difference between defining a function and calling a function.

1 Like
14

The point is that, in my example, running the function does not throw an error. In the context of my example, the following throws an error,

function Foo{T}(::UndefInitializer, n::Int) where {T<:Real}
	d = fld(n * (n + 1), 2)
	A = Matrix{T}(undef, d, d)
	return Foo(A, n)
end

Foo{Float64}(undef, 2);
# => UndefVarError: T not defined

but the following does not:

function Foo{T}(::UndefInitializer, n::Int) where {T<:Real}
	d = fld(n * (n + 1), 2)
	A = zeros(d, d)
	return Foo(A, n)
end

Foo{Float64}(undef, 2);
# works fine
15

The relevant section of the manual is this one. As @lmiq pointed out, parametric constructors are kinda special, they are the only kinda of function (or function-like?) that can be passed explicit types (without them being a positional argument).

As @lmiq also pointed out, you can put type restrictions directly in the struct definition (this is covered by the link to the manual I provided, and the manual explains how this is equivalent to automatically defining a constructor with a where clause).

Carefully reading this section I noticed one thing. No example in the whole section involves defining an outer constructor (i.e., a constructor outside the struct scope) that had the {T, ...} clause between the function name and the opening parenthesis. The only examples of constructor definitions with this syntax are of inner constructors.

However, as was pointed out in this thread, there is code that defines outer constructors that receive explicit type parameters and work anyway. I agree with @CameronBieganek we can only guess is that there is some kind of ambiguity problem that arises in some situations but not in others.

I would bet on the following: on the minimal example provided by @lmiq the constructor has no way to guess what is the type T. Foo{T}(::UndefInitializer) calls Foo(Vector{T}(undef,2)), what Foo (with no type parameters) will guess to be T? It received a Vector{T}, the struct has only a field x :: S but it is parametrized as Foo{T,S <: Vector{<:T}}, the Vector{T} allow us to assert that S == Vector{T}, but we cannot guess what T (the struct parameter) is, because if it is a Vector{Int} that is being passed (i.e., S == Vector{Int}), we have that Int <: T and, therefore, T can be many things, like Integer or Number. The problem does not happen if T is fixed (instead of being any supertype) because then it is clear that T can only be Int.

2 Likes
16

I think our terminology is different. When I said running a function, I meant calling a function. Where you said running a function, I said defining a function. :slightly_smiling_face:

This behavior is necessary so that you can define your functions in any order, and so that you can define mutually recursive functions. For example, you can define functions “out of order” like this:

julia> foo() = bar()
foo (generic function with 1 method)

julia> foo()
ERROR: UndefVarError: bar not defined

julia> bar() = 1
bar (generic function with 1 method)

julia> foo()
1

The order doesn’t matter because Julia doesn’t check whether internal variables exist yet when you first define a function.

1 Like
17

Yeah, I mixed up some words there! Thanks for pointing that out :laughing: :sweat_smile:.

18

No worries. I should probably stick to “call” rather than “run”, since it’s more precise.

1 Like
19

Excellent explanation! Now I get it!

I basically disconnected T and S when I defined Foo, and now my constructor requires T to be retrieved from S. That was hard to get my head around that :sweat_smile:

20

Good recapitulation, I was worried that it was not intelligible because T is used both in the struct parameter and in the constructors with different meaning (on that MWE).

However, this is only relevant for the MWE, because for me, with Julia 1.5.3, your original example works without any changes except removing <: AbstractArray{T, 4} from the struct (because your inheritance makes a specialized Base.show trigger and it expects that you had defined a Base.size for your type to be able to print it). So I think your original code does not make the same mistake intentionally created by @lmiq to trigger the same error message. Julia 1.4.1 probably had a bug that triggered the same error on situations where it did not apply.

1 Like