How to understand interface requirements of standard library

New to Julia
1

Concrete problem: I have a ConcreteVector implementing AbstractVector with some Base. methods. Calling Base.vcat returns a Vector, but I would like it to return a ConcreteVector. So either I’m missing some necessary methods or I have to implement Base.vcat myself for this type or I have to do a conversion. Is there other documentation besides Arrays · The Julia Language?

Abstract problem: in general I don’t have a clue where to start with this kind of problems besides maybe debugging the standard library.

How do you solve similar problems?

2

you probably need to post some code for this to be answer-able

3

The code

struct ConcreteVector{T} <: AbstractVector{T}
    vector::Vector{T}
end

Base.size(v::ConcreteVector{T}) where T = size(v.vector)
Base.copy(v::ConcreteVector{T}) where T = ConcreteVector{T}(copy(v.vector))

Base.push!(v::ConcreteVector{T}, t::T) where T = push!(v.vector, t)

Base.getindex(v::ConcreteVector{T}, i::Int) where T = getindex(v.vector, i)

Base.empty!(v::ConcreteVector{T}) where T = empty!(v.vector)

Base.iterate(v::ConcreteVector{T}, state) where T =
    iterate(v.vector, state)
Base.iterate(v::ConcreteVector{T}) where T =
    iterate(v.vector)

v = ConcreteVector{Int}(Vector{Int}())
push!(v, 1)
vcat(v, 2)

returns

2-element Vector{Int64}:
 1
 2
4

you should just use Vector, btw some of these are not quite right:

Base.empty!(v::ConcreteVector{T}) where T = begin empty!(v.vector); return v end

Base.push!(v::ConcreteVector{T}, t::T) where T = begin push!(v.vector, t); return v end

In this artificial case I would suggest defining:

julia> Base.vcat(v::T, x...) where T<:ConcreteVector = T(vcat(v.vector, x...))

julia> vcat(v, 2)
2-element ConcreteVector{Int64}:
 1
 2

# you might want to make `eltype` promotion happen
julia> Base.vcat(xs::T...) where T<:ConcreteVector = T(mapreduce(e->e.vector, vcat, xs))

julia> vcat(v, v)
2-element ConcreteVector{Int64}:
 1
 1
5

Hi Jerry, thanks for your help.

you should just use Vector

This is just a simplification for demonstration purposes.

julia> Base.vcat(v::T, x...) where T<:ConcreteVector = T(vcat(v.vector, x...))

This is what I feared: it would mean implementing every method in the standard library usable for AbstractVectors. How many might that be?

6

just implement enough to pass your unit tests, and your unit tests should reflect how your vector can be used.

the alternative is to have a formal interface and then put 50% junk as filler. I’d rather have this duck-interfacing, very practical.

also checkout: GitHub - JuliaArrays/ArrayInterface.jl: Designs for new Base array interface primitives, used widely through scientific machine learning (SciML) and other organizations

7

So I started digging into the library. Here is cat_t from abstractarray.jl

@inline cat_t(::Type{T}, X...; dims) where {T} = _cat_t(dims, T, X...)
@inline function _cat_t(dims, ::Type{T}, X...) where {T}
    catdims = dims2cat(dims)
    shape = cat_size_shape(catdims, X...)
    A = cat_similar(X[1], T, shape)
    if count(!iszero, catdims)::Int > 1
        fill!(A, zero(T))
    end
    return __cat(A, shape, catdims, X...)
end

It hints to

cat_similar(A, ::Type{T}, shape) where T = Array{T}(undef, shape)
cat_similar(A::AbstractArray, ::Type{T}, shape) where T = similar(A, T, shape)

So implementing similar (resolving some ambiguities) and setindex! we get

struct ConcreteVector{T} <: AbstractVector{T}
    vector::Vector{T}
end

Base.size(v::ConcreteVector{T}) where T = size(v.vector)
Base.copy(v::ConcreteVector{T}) where T = ConcreteVector{T}(copy(v.vector))
Base.similar(v::ConcreteVector, ::Type{T}, dims::Tuple{Int64}) where T =
    ConcreteVector{T}(Vector{T}(undef, dims[1]))

Base.push!(v::ConcreteVector{T}, t::T) where T = push!(v.vector, t)

Base.getindex(v::ConcreteVector{T}, i::Int) where T = getindex(v.vector, i)
Base.setindex!(v::ConcreteVector{T}, t::T, i::Int) where T = setindex!(v.vector, t, i)

Base.empty!(v::ConcreteVector{T}) where T = empty!(v.vector)

Base.iterate(v::ConcreteVector{T}, state) where T =
    iterate(v.vector, state)
Base.iterate(v::ConcreteVector{T}) where T =
    iterate(v.vector)

v = ConcreteVector{Int}(Vector{Int}())
push!(v, 1)
println(typeof(v))
v = vcat(v, 2)
println(typeof(v))

resulting in

ConcreteVector{Int64}
ConcreteVector{Int64}

So if noone has a better idea it seems stepping into the standard library is the way to go.

1 Like
8

https://docs.julialang.org/en/v1/manual/interfaces/#man-interface-array

these are documented, setindex!() was a oversight on my part, not sure if it’s easier or harder than look at the base.

1 Like
9

Good point!

I mostly ignored the optional methods and simply didn’t connect vcat and similar.