@parallel vcat throws TypeError but @parallel hcat does not

General Usage
,
1

I think I’m having the same issue as this post: I get an error when I use @parallel vcat where each iteration returns a Dict{Symbol, Array{Float64}} , but not when I use @parallel hcat.

Suppose I define the following functions:

function parallel_hcat(make_dict::Function)
    res = @sync @parallel hcat for i = 1:10 
        make_dict()
    end
    println(typeof(res))
end

function parallel_vcat(make_dict::Function)
    res = @sync @parallel vcat for i = 1:10 
        make_dict()
    end
    println(typeof(res))
end

Both parallel_vcat and parallel_hcat work as expected when I specify that the dictionary values are 1-dimensional arrays:

julia> addprocs(3);

julia> @everywhere make_dict_spec() = Dict{Symbol, Array{Float64, 1}}()

julia> parallel_hcat(make_dict_spec)
Array{Dict{Symbol,Array{Float64,1}},2}

julia> parallel_vcat(make_dict_spec)
Array{Dict{Symbol,Array{Float64,1}},1}

However, only parallel_hcat works when I don’t specify the array dimension:

julia @everywhere make_dict_unspec() = Dict{Symbol, Array{Float64}}()

julia> parallel_hcat(make_dict_unspec)
Array{Dict{Symbol,Array{Float64,N}},2}

julia> parallel_vcat(make_dict_unspec)
ERROR: TypeError: collect_to!: in typeassert, expected Array{Dict{Symbol,Array{Float64,N}},1}, got Array{Dict{Symbol,Array{Float64,N}},1}
 in collect_to!(::Array{Array{Dict{Symbol,Array{Float64,N}},1},1}, ::Base.Generator{Array{Task,1},Base.#wait}, ::Int64, ::Int64) at ./array.jl:343
 in collect_to_with_first!(::Array{Array{Dict{Symbol,Array{Float64,N}},1},1}, ::Array{Dict{Symbol,Array{Float64,N}},1}, ::Base.Generator{Array{Task,1},Base.#wait}, ::Int64) at ./array.jl:327
 in collect(::Base.Generator{Array{Task,1},Base.#wait}) at ./array.jl:308
 in preduce(::Function, ::Function, ::UnitRange{Int64}) at ./multi.jl:2013
 in macro expansion at ./task.jl:326 [inlined]
 in parallel_vcat(::#make_dict_unspec) at ./REPL[11]:2

Is there a reason for this asymmetric behavior between @parallel hcat and @parallel vcat? What does the error

ERROR: TypeError: collect_to!: in typeassert, expected Array{Dict{Symbol,Array{Float64,N}},1}, got Array{Dict{Symbol,Array{Float64,N}},1}

mean?

1 Like