Problem Adding a dimesion to array/vector

General Usage
, , ,
1

Hello, i have a
Vector{Any} with 25701 elements made of: 100×40 Array{Float64,2}

id like to add a dimension to it so from 25701 * 100 * 40*1

i have tried like written here
https://stackoverflow.com/questions/42312319/how-do-i-add-a-dimension-to-an-array-opposite-of-squeeze

and here:
https://stackoverflow.com/questions/58217442/adding-an-additional-dimension-to-an-array

but the result everytime is:

25701×1 Array{Any,2}:
[0.36919015509523057 0.3868927477017365 … 0.0 0.0; 0.36526140799820705 0.37552196118488257 … 0.0 0.0;

which is not what i need

i need it for a conv2d using keras
im pretty sure in python is like this: x.reshape(x.shape + (1,))

2

Sounds like you’re not trying to add a dimension at all, but instead convert a 1D array of 2D arrays into a 4D array, with the first dimension corresponding to the outer array, and the last dimension being 1. The following isn’t pretty and it isn’t general, but it will do exactly that:

function toconvkernel(A::AbstractArray{<:AbstractMatrix})
    s = size(first(A))
    @assert all(==(s), size.(A))

    B = Array{eltype(eltype(A)), 4}(undef, length(A), s..., 1)
    for (i, a) in enumerate(A)
        B[i, :, :, 1] = a
    end
    B
end
1 Like
3

For something that looks a little closer to the Python implementation, you might try

toconvkernel(A) = mapreduce(x->reshape(x, 1, size(x)..., 1), vcat, A)

Use reshape to convert each element of the outer array to a 4-dimensional array, then concatenate them along the first axis.

2 Likes
4

It did not occur to me before, but probably the best way is with Tullio

using Tullio
@tullio B[i, j, k, 1] := $A[i][j, k]

@contradict Note that the mapreduce way unfortunately scales very poorly due to repeated allocation (19s vs 14ms for a 1000x100x40x1 example)

5

@tomerarnon

hello, thank you but its not working
i tried with:
LoadError: “can’t use index 25701 on LHS for a new array”
@tullio B[25701, 100, 40, 1] := $x[25701][100, 40]

but it would be way better if it is a faster method compared to the other 2

the first returns:
MethodError: no method matching toconvkernel(::Array{Any,1})

@contradict
even though it is slow and occupy a lot of memory it is working, thanks:
result: 25701×100×40×1 Array{Float64,4}:

6

There is a package with functions doing exactly this: https://github.com/JuliaData/SplitApplyCombine.jl. Specifically, you need combinedims or combinedimsview function from there.

1 Like
7

The index letters in these einsum-like expressions are loop variables, not sizes. It should run literally as written (although without the dollar sign) if A is your vector of matrices:

Ordering things @tullio B[j, k, 1, i] := A[i][j, k] should be faster – the last index of Julia’s arrays is the biggest step in memory.

There are also many ways to make array-like views of all the slices, LazyStack.jl or JuliennedArrays.jl are two small packages for exactly this, and SplitApplyCombine.jl as mentioned seems to also include this.

1 Like
8

I defined toconvkernel as taking in an array with element type <:AbstractMatrix and you seem to have Array{Any, 1} (the eltype is Any) instead. You can remove the type annotation from the function definition, but it would be better to have your array not be an array of Any in the first place. Wherever you create that, it would be better to make sure it is strictly typed.