The broadcastcall syntax for function with multiple return values results in an array of tuples.
Sometimes, you want the tuple of arrays instead. This comes up when computing
multiple related values on a multidimensional grid. I think I’ve come up with a good solution, using macros in Cartesian. However, this is the first time I’m using some of Julia’s features, and I have a few questions.
The problem:
julia> f(a, b) = a+b, a*b, ab; # some transform returning multiple values
julia> v = f.(rand(3,1), rand(1,4));
julia> typeof(v)
Array{Tuple{Float64,Float64,Float64},2}
I’d need to store the 3 return values in 3 arrays:
julia> x, y, z = unpack(v);
julia> x
3×4 Array{Float64,2}:
0.301013 0.888299 1.03866 1.0867
0.853248 1.44053 1.5909 1.63894
0.687546 1.27483 1.4252 1.47324
See: Destructuring and broadcast
The solution
After stumbling on the Base.Cartesian
package through FastConv.jl
(and getting my mind blown), I figured out the following:
@generated function unpack{N,M,U}(v::Array{T,N} where T <: NTuple{M,U})
quote
@nexprs $M i > out_i = similar(v,U)
@inbounds @nloops $N j v begin
@nexprs $M i > ((@nref $N out_i j) = (@nref $N v j)[i])
end
return @ntuple $M out
end
end
Which works correctly, and is faster than other implementations using stuff like first.(v)
.
julia> v = f.(rand(500,1,1), rand(1,500,500));
julia> @time unpack(v);
1.376500 seconds (11 allocations: 2.794 GiB, 4.19% gc time)
julia> x, y, z = f.(rand(100,1,1), rand(1,100,100)) > unpack; # hurray!
I figured out it can be useful for others: https://github.com/spalato/Unpack.jl
Questions

What exactly is going on in the type annotations?
I couldn’t seem to find the appropriate documentation forwhere
. Specifically, why doeswhere T <: NTuple{M,U}
need to be in the parenthesis, while{N,M,U}
seem to annotate the function? 
I was somehow expecting the
function unpack{N,M,U}(...)
to provide multiple methods for different values of{N,M,U}
. This is apparently not the case. Querying the methods always return the following, no matter how many different argument combinations where called before. Does the{N,M,U}
annotation have a purpose beyond defining the variables?julia> methods(unpack) # 1 method for generic function "unpack": unpack(v::Array{T,N} where T<:Tuple{Vararg{U,M}}) where {N, M, U} in Unpack at ...

Are all these type annotations really necessary, or am I overdoing this?

Currently, this handles only tuples with elements of the same type (
NTuples
). This is usually the case in my line of work. Is there a way to obtain the types of the tuple from the typeT
, to generalize this? 
This implementation requires the allocation of an intermediary array (
v
in previous examples). This thus uses twice the memory compared to a manual loop. Is there a way to avoid this intermediary allocation (say, ‘hijack’ the broadcast call)?
Cheers!