AbstractArray vs. Array in Julia types

question
performance

#1

Consider the simple type:

type Foo{T}
A::AbstractArray{T} # (1)
end

# (2) update Foo.A with some operation on external array A
update{T}(foo::Foo{T}, A::AbstractArray{T}) = foo.A = 2*A

I created two marks in the code (1) and (2) and I would like to ask you what are the best practices regarding the choice AbstractArray vs. Array in Julia.

I understand that by replacing AbstractArray with a dense Array in both (1) and (2) we make the type concrete and the compiler can perform all sorts of memory optimizations to store the arrays in contiguous memory chunks. I wonder how we should reason through this tradeoff between generic functions that accept any type of array and efficient memory layouts with dense arrays. Could you please share your thoughts? Specifically for (1) and (2), how do you decide between using AbstractArray vs. Array?


#2
julia> struct Foo{A_t <: AbstractArray{T} where T}
           A::A_t
       end

julia> Foo(1:10)
Foo{UnitRange{Int64}}(1:10)

julia> Foo(rand(3))
Foo{Array{Float64,1}}([0.118515, 0.232194, 0.177065])

#3

If you just make it parametric on the type of array

type Foo{T<:AbstractArray}
A::T
end

then you don’t lose performance, and on v0.6 you can do this recursively, so

type Foo{T<:AbstractArray{T2} where T2}
A::T
end

(or something like that, I’m not too familiar with v0.6 where syntax yet but I know you can do this).

Generally your algorithm is compatible with AbstractArray if it’s compatible with Array. Pretty much the only case where that’s not true is when you’re making a call to C/Fortran which expects a contiguous array. Otherwise, if it’s pure Julia code, you can usually expect the array type to do all of the “normal stuff”, like have a linear index and broadcast mathematical expressions. Then you leave it to the user to give you a GPUArray or whatnot.


#4

Perhaps something like:

julia> struct Foo{T, N, A_t <: AbstractArray{T,N}} <: AbstractArray{T,N}
           A::A_t
       end

julia> Foo(v::AbstractArray{T, N}) where{T, N} = Foo{T, N, typeof(v)}(v) # Aux constructor

also works?


#5

Thanks @kristoffer.carlsson, so I think I will require Julia v0.6 in my package to enjoy both generic and performant code.


#6

Thanks @ChrisRackauckas, I think I will have to go with Julia v0.6 then.


#7

@ChrisRackauckas, @kristoffer.carlsson

By the way, how does one read up on the upcoming features for the version 0.6 in one place? I’ve seen voluminous discussions in different places, but often it is not quite clear what will eventually make it and the documentation on the Julia language website only seems to refer to the 0.5 (?) version.

Thanks,

Petr


#8

NEWS.md contains major changes. https://github.com/JuliaLang/julia/blob/release-0.6/NEWS.md


#9

Fantastic. Many thanks.


#10

@PetrKryslUCSD, I believe that the development deviated from the original plans regarding Julia v0.6. We don’t have documentation nor official release on the downloads page. The nightly builds are already targeting Julia v0.7, my guess is that the next official release will be either v0.7 or v1.0.


#11

Right, thanks for the heads up.


#12

That is false. The downloads page has a release candidate for v0.6 (currently release candidate 2: v0.6-rc2). When “all of the bugs are fixed” there, that will be the next release: v0.6. The changes are described in the news page @mbauman linked, and the new Documenter.jl docs will become the standard docs.


#13

Please be more careful with the information you spread. This is blatantly false.


#14

@ChrisRackauckas thanks for pointing out the release candidates, these I know exist. They are there for a while. I remember the planned deadlines to release v0.6 have passed, please correct me if I am wrong. Anyways, I wish I had the documentation for v0.6 as well.


#15

https://docs.julialang.org/en/latest/


#16

Hi @kristoffer.carlsson, isn’t latest referring to v0.7? Maybe my confusion is because I thought latest and nightly were the same thing.