Broadcasting arrays with different number of dimension

General Usage
,
1

In PyTorch it’s possible to do the operation:

# v.shape = torch.Size([1, 2, 1])
# b = torch.Size([2, 8])
a = v - b
# a.shape = torch.Size([1, 2, 8])

However when I try to implement this in Julia
I get an error

rand(1, 2, 1)-rand(2,8)

DimensionMismatch("dimensions must match: a has dims (Base.OneTo(1), Base.OneTo(2), Base.OneTo(1)), b has dims (Base.OneTo(2), Base.OneTo(8)), mismatch at 1")

Stacktrace:
 [1] promote_shape at ./indices.jl:178 [inlined]
 [2] promote_shape(::Array{Float64,3}, ::Array{Float64,2}) at ./indices.jl:169
 [3] -(::Array{Float64,3}, ::Array{Float64,2}) at ./arraymath.jl:38
 [4] top-level scope at In[318]:1
 [5] include_string(::Function, ::Module, ::String, ::String) at ./loading.jl:1091

What is the right way to do this?

2

What is the right way to do this?

That depends. What is it supposed to do?

1 Like
3

First of all you have to use broadcasting, that is, dot the minus sign:

v .- b

This still won’t work, since the second dimension of the arrays are 2 and 8, respectively. They are incompatible, so maybe you can explain what pytorch is doing, it doesn’t make sense to me.

2 Likes
4

Python has row major storage, so it matches the dimensions from the right. Julia has column major storage, but matching the same dimensions from the left doesn’t work. You can permute the dimensions for example.

1 Like
5

That is mindbending🤯 But ‘row-major’ needs a new name. ‘Last-dimension-major’, or something.

1 Like
6

I griped about that once, last dimension contiguous was suggested

1 Like
7

The easiest way is just to drop the unit dimensions of v before broadcasting, either by constructing v as a 1d array to start with or by calling dropdims (which is essentially free because it just calls reshape under the hood, sharing data):

julia> dropdims(v, dims=3) .- b

julia> dropdims(v, dims=(1,3)) .- b
2×8 Matrix{Float64}:
 -0.178031  -0.593356  -0.885534  …  -0.526637  -0.0416247  -0.579015
 -0.368148  -0.883498  -0.693581     -0.372224  -0.708516   -0.522531
3 Likes
8

The first dimenstion of rand(1,2,1) is supposed to be batch size.
I’m currently using this for test

b = reshape(rand(2,8), (1, 2, 8))
rand(1,2,1) .- b

If it breaks I’ll try methods suggested here,
thanks for support