Partitioning a matrix in batches

New to Julia
1

Hi there,

Let’s say I have a matrix of size 20x10. What is the best way to partition this matrix into batches of 20x2? In an ideal world I would like to have them structured as Array{Array{Float32,2},1}, leading to one row per batch. But I can not seem to get this working…

Here’s what I had in mind

matrix = zeros(20, 10)
batchsize = 2
result = Array{Array{Float32,2},1}

for i in 1:batchsize:size(matrix, 2)
        first_col = i
        last_col = i + batchsize - 1

        if last_col > size(matrix, 2)
                batch_i_matrix = matrix[:, first_col:size(matrix, 2)]
        else
                batch_i_matrix = matrix[:,first_col:last_col]
        end

        append!(result, batch_i_matrix)
end

So everything here works fine up until where I want to “store” the sub matrices. The append! methods is telling me there is a MethodError:

ERROR: MethodError: no method matching append!(::Type{Array{Array{Float32,2},1}}, ::Array{Float64,2})
Closest candidates are:
  append!(::BitArray{1}, ::Any) at bitarray.jl:766
  append!(::DataFrames.DataFrame, ::Any; cols, promote) at /home/wuyts/.julia/packages/DataFrames/3ZmR2/src/other/tables.jl:50
  append!(::PooledArrays.PooledArray{T,R,1,RA} where RA where R where T, ::AbstractArray) at /home/wuyts/.julia/packages/PooledArrays/yiLq3/src/PooledArrays.jl:385
  ...
Stacktrace:
 [1] top-level scope at ./REPL[144]:11

After this error I thought that maybe changing this append! statement to append!(result, [batch_i_matrix]), which returns the exact same type as the type I want to append to, but no luck.

Any suggestions?

Many thanks,
Sander

2

I think you want push! not append!, but note that result = Array{Array{Float32,2},1} is a type, result = Array{Array{Float32,2},1}() is an empty vector (although still has different eltype).

But you can also do this:

[matrix[:, i:min(i+batchsize-1,size(matrix,2))] for i in 1:batchsize:size(matrix,2)]

or this

map(Iterators.partition(axes(matrix,2), batchsize)) do cols
    matrix[:, cols]
end
1 Like
3

Wow thank you!

That sure is a lot more elegant than what I have been writing. Your first suggestion was easier to understand than the second one, at first sight, but now both make sense to me.

Initiating with result = Array{Array{Float32,2},1}() and changing append!() to push! also works.

Thank you!

1 Like