How to get the container type of a container?

[singularitti]

For example, I have an x = Vector{Int}, then do what operations on x can I get the Vector ?

[tkf]

If you want to get Array from Vector{Int}, I’ve been using

constructor_of(::Type{T}) where T =
    getfield(T.name.module, Symbol(T.name.name))

since Julia 0.6 for this purpose. If you only target 1.x, I think

constructor_of(::Type{T}) where T =
    getfield(parentmodule(T), Symbol(Base.typename(T)))

may be better.

But getting Vector from Vector{Int} seems harder because Vector{Int} actually is Array{Int,1}.

If you need to special-case only a few types, you can add a method to constructor_of:

constructor_of(::Type{T}) where {T <: Vector} = Vector

[c42f]

I’m not sure exactly what you’re asking here.

Do you have a vector instance (eg, x = [1,2,3]) and you want to get the type? You can use typeof(x) for that.

Do you have a vector instance x and want to create another vector y which is the same shape and type? Use similar(x) for that.

[singularitti]

Sorry, I should be more clear. I am writing a pseudo-code here because it is not a valid syntax in Julia:

julia> f(x::T{Int}) where {T <: AbstractArray} = convert(T{Float64}, x)
ERROR: TypeError: in Type{...} expression, expected UnionAll, got TypeVar

i.e., f eats an AbstractArray of something (Int here), and converts it to an AbstractArray of another thing (Float64 here). However, Julia does not allow TypeVar in the function definition. I know I can use map in such simple case, but what I am actually facing is a bit more complicated that this example so I feel hard to write map.

[Tamas_Papp]

@singularitti

f eats an AbstractArray of something ( Int here), and converts it to an AbstractArray of another thing ( Float64 here).

I am not sure it would work in general, since convert does not need to be defined for a type to implement the AbstractArray interface. Eg

struct MyArray{T, N, A <: AbstractArray{T,N}} <: AbstractArray{T,N}
    parent::A
end

Base.size(A::MyArray) = size(A.parent)
Base.getindex(A::MyArray, I...) = getindex(A.parent, I...)

A = rand(Int,3,3)
B = MyArray(A)
convert(MyArray{Float64}, B)    # will error

[mauro3]

If I understand correctly, you want to strip the type parameters from a type. That is not possible in general Stripping parameter from parametric types. similar is made for this.

[singularitti]

Yes, you are right. That’s what I want!

[MilesCranmer]

I’ve landed on this thread many, many times so I’ll just paste my modified solution here. The above version was not general enough for me, since I wanted to get the container of an arbitrary user-defined type, not just AbstractArrays.

I don’t remember where I saw this recursive solution originally (I think maybe the Julia docs, evolved from this post?), but here is the code I’ve been using, with some modifications to make it instantaneous

function _container_type(T::Type)
    isa(T, UnionAll) && return _container_type(T.body)
    return T.name.wrapper
end

@generated function container_type(::Type{T}) where {T}
    container = _container_type(T)
    return :($container)
end

The reason it’s inside a @generated is just for caching. You can’t do (::Type{T}) where {T} in the _container_type because Julia will hit a stack overflow.

With this, you can get the container for arbitrary types, with no runtime cost:

julia> container_type(AbstractArray)
AbstractArray

julia> container_type(AbstractArray{Float64,3})
AbstractArray

julia> struct F{T1,T2,T3}
           x::T1
           y::T2
           z::T3
       end

julia> container_type(F{Float64,String,Tuple{Float64,Float32}})
F

julia> @btime container_type(F{Float64,String})
  0.875 ns (0 allocations: 0 bytes)
F

However, this will breaks if there is some abstract type with more type parameters than the user-defined struct.

[Henrique_Becker]

Are the fields of DataType objects part of the public interface of Julia?

[MilesCranmer]

Doesn’t seem like there’s an alternative:

@liuyxpp

Base.typename returns a Core.TypeName object. To get the bare type, you still have to use Base.typename(T).wrapper. It seems useing T.name.wrapper is more straightforward.

[MilesCranmer]

Actually, I guess this is a bit cleaner?

function _container_type(T::Type)
    return Base.typename(T).wrapper
end

@generated function container_type(::Type{T}) where {T}
    container = _container_type(T)
    return :($container)
end

But .wrapper is still needed

[MilesCranmer]

Oops, now the @generated isn’t even needed.

function container_type(::Type{T}) where {T}
    return Base.typename(T).wrapper
end

[aplavin]

For comparison, ConstructionBase uses getfield(parentmodule(T), nameof(T)) for this purpose: ConstructionBase.jl/src/ConstructionBase.jl at master · JuliaObjects/ConstructionBase.jl · GitHub.

[MilesCranmer]

Thanks!

I have no idea how to evaluate these because they both compiled to the same LLVM…

julia> function container_type(::Type{T}) where {T}
           return Base.typename(T).wrapper
       end;

julia> @generated function constructorof(::Type{T}) where T
           getfield(parentmodule(T), nameof(T))
       end

which gives

julia> @code_llvm container_type(F{Float64,Float32})
;  @ REPL[12]:1 within `container_type`
define nonnull {}* @julia_container_type_478({}* readonly %0) #0 {
top:
;  @ REPL[12]:2 within `container_type`
  ret {}* inttoptr (i64 4534794992 to {}*)
}

julia> @code_llvm constructorof(F{Float64,Float32})
;  @ REPL[11]:1 within `constructorof`
define nonnull {}* @julia_constructorof_475({}* readonly %0) #0 {
top:
; ┌ @ REPL[11]:1 within `macro expansion`
   ret {}* inttoptr (i64 4534794992 to {}*)
; └
}

The downside of constructorof is that uses a generated function, and the downside of container_type is that it doesn’t use the public interface… sigh…

[nsajko]

I’ll note that Polynomials.jl, for example, also uses the Base.typename(T).wrapper solution:

JuliaMath/Polynomials.jl/blob/2de280b8095b28d15cf310ccbe37c6b73211d849/src/contrib.jl#L137-L149

However, the Julia Manual actually addresses this question, and the recommended solution is completely different, and much nicer IMO:

https://docs.julialang.org/en/v1/manual/methods/#Building-a-similar-type-with-a-different-type-parameter

@Julia Manual

Building a similar type with a different type parameter

When building generic code, there is often a need for constructing a similar object with some change made to the layout of the type, also necessitating a change of the type parameters. For instance, you might have some sort of abstract array with an arbitrary element type and want to write your computation on it with a specific element type. We must implement a method for each AbstractArray{T} subtype that describes how to compute this type transform. There is no general transform of one subtype into another subtype with a different parameter.

So: create a function, and then have each relevant type implement a method of the function. This requires some thinking about interface design, though.

[MilesCranmer]

@nsajko

I’ll note that Polynomials.jl, for example, also uses the Base.typename(T).wrapper solution:

Thanks for pointing this out; it’s good to know that type of approach is at least robust enough to be used in larger packages.

@nsajko

Building a similar type with a different type parameter

When building generic code, there is often a need for constructing a similar object with some change made to the layout of the type, also necessitating a change of the type parameters. For instance, you might have some sort of abstract array with an arbitrary element type and want to write your computation on it with a specific element type. We must implement a method for each AbstractArray{T} subtype that describes how to compute this type transform. There is no general transform of one subtype into another subtype with a different parameter.

So: create a function, and then have each relevant type implement a method of the function. This requires some thinking about interface design, though.

I suppose, but then again, I would think the main motivation for automatically extracting a container type is to avoid burdening the user with a method they need to implement. (At least this was true for my case). This is especially true if the methods are further upstream in the dependency tree.

I guess the strategy imposed by ConstructionBase.jl satisfies both of these desires? Because it (1) by default just returns the container type (which is good enough 99% of the time), but (2) enables the user to write a new constructorof method for any objects that require a weird constructor:

(From Home · ConstructionBase.jl)

constructorof(T::Type) -> constructor

Return an object constructor that can be used to construct objects of type T
from their field values. Typically constructor will be the type T with all parameters removed:

julia> using ConstructionBase

julia> struct T{A,B}
           a::A
           b::B
       end

julia> constructorof(T{Int,Int})
T

It is however not guaranteed, that constructor is a type at all:

julia> struct S
           a
           b
           checksum
           S(a,b) = new(a,b,a+b)
       end

julia> ConstructionBase.constructorof(::Type{<:S}) =
           (a, b, checksum=a+b) -> (@assert a+b == checksum; S(a,b))

julia> constructorof(S)(1,2)
S(1, 2, 3)

julia> constructorof(S)(1,2,4)
ERROR: AssertionError: a + b == checksum

Instead constructor can be any object that satisfies the following properties:

• It must be possible to reconstruct an object from the elements of getfields:

ctor = constructorof(typeof(obj))
@assert obj == ctor(getfields(obj)...)
@assert typeof(obj) == typeof(ctor(getfields(obj)...))

(Though in this example I don’t follow why the user wouldn’t want to simply define S(a, b, checksum) = ... at the source)

[aplavin]

@MilesCranmer

I have no idea how to evaluate these because they both compiled to the same LLVM…

Only on recent Julia, while for example on 1.6 only the parentmodule... solution is zero-cost.
I guess that’s why ConstructionBase uses it, otherwise it would make sense to get rid of this @generated function.