Understand vectorized dot operators

Dear folks

I’m an old Matlab user who is trying to switch to Julia. In Matlab, I use array (element-wise) arithmetic a lot.

When I tried to make element-wise arithmetic using Julia, I noticed the following strange behavior:

  • Adding a scalar to a vector needs dot “.+”, while multiplying them works fine without the dot!!!
  • Multiplying two vectors needs dot “.*”, while adding them works fine without the dot!!!

I got confused!! Please help me understand when I need the dot and when I don’t need it to make array arithmetic.

One more thing, in Matlab, array arithmetic works with compatible size arrays. Is this available in Julia?

Thanks and regards.

That’s just maths:

Also, multiplying vectors does not need a dot, but it needs dimensions to be consistent to either get an inner (dot) product or an outer product:

julia> v = rand(3);

julia> v * v
ERROR: MethodError: no method matching *(::Vector{Float64}, ::Vector{Float64})

julia> v' * v
0.24979883493928956

julia> v * v'
3×3 Matrix{Float64}:
 2.91655e-5   0.00266555  0.000423667
 0.00266555   0.243615    0.0387206
 0.000423667  0.0387206   0.00615433
1 Like

Standard linear-algebra operations (e.g. adding/subtracting vectors or other arrays of the same size or multiplying them by scalars, or performing valid matrix/vector multiplications) does not use a dot. Any other elementwise operation requires a dot.

1 Like

Thanks for your answer, but please help me understand the rule. Matlab rules are so clear.

Why the operation works with addition but fails with multiplication? Why the rule applies to one operator and not on the other??

Welcome to the Julia forum!

I feel the Julia rules are clearer (and I also once came from Matlab), because you nearly never “accidentally broadcast” which happened to me in Matlab often and I suddenly hat matrices where I just expected vectors. So maybe to extend on Stevens answer.

Assume you have a vector like

x = zeros(3)
y = ones(3)

they are of same size so we can easily add them with x+y but you can even say, please apply plus to both vectors element wise by saying x .+ y that is – in the end – the same

for adding a scalar, say a=1 the types are not equal. One is a number one is a vector. So then we have to clarify what we mean by number+vector (which per se is not defined. But if the think of .+ as “please add elementwise” a .+ x works fine (please add a to each element of x).

For multiplication, scaling a vector by a factor is well-defined – vectors form a vector space so that’s why a*x works, but similar to adding two vectors (or the .+ for scalars) you can also say “please scale every element of x by a” that is a .* x. The thing here is that scaling a vector (ax) is defined as scaling every element of x by a, so that is why that works both the same again (similar to adding vectors).

2 Likes

I thought that’s what my (and Steven’s) answer had done - when an operation is meaningfully defined mathematically it works, if it isn’t, it doesn’t.

As explained in the first link I shared, adding a scalar to a vector is not a well defined operation. Therefore, scalar + vector will aise a method error. If what you meant is “actually add the scalar to each element of the vector” then you need to be explicit and broadcast the addition by using .+.

Adding two vectors of the same length is a well defined operation, so it will work without using an elementwise addition operator (although in this case the result is the same with or without broadcasting).

Vector multiplication is again a well defined concept, but you need to make sure the dimensions are right. You can think of Julia vectors as column vectors, so v = rand(3) will create a 3-by-1 vector. You therefore cannot do v * v as that would mean multipling (3 \times 1) \times (3 \times 1) which does not work. You can do v' * v which multiplies (1 \times 3) \times (3 \times 1) to yield a (1 \times 1) (scalar) output, and you can do v * v' which gives a (3 \times 1) \times (1 \times 3) = (3 \times 3) matrix.

If you actually meant “multiply each element of v by itself” you again need to be explicit and say v .* v

2 Likes

I wouldn’t phrase it this way. Julia’s vector .+ scalar is perfectly well defined and meaningful mathematically, and symbols can mean whatever we want them to mean. It’s just not an operation of linear algebra, and Julia reserves + for the ordinary algebraic meaning.

Julia used to be more Matlab-like and supported implicit vectorization/broadcasting of many functions (e.g. vector + scalar, sqrt(vector), and so forth). However, this was changed when we got the f.(x) “dot syntax” to broadcast any function, since it turns out that this gave us big advantages by being syntactically explicit, as explained in this blog post.

Interestingly, Julia originally had vector + scalar, then it was removed (julia#5810), then it was added back (julia#7226), and then finally it was removed again (julia#22932) once we had fusing f.(x) syntax for all functions. See the linked issues for long discussions of the pros/cons — this is not something we decided on a whim, but after extensive debate and trial-and-error.

See also Idea Julia Lite or 'Juliette'? - #55 by stevengj

4 Likes

I’m not so sure about that (Matlab code):

>> x = rand(3,1); y = rand(3,1);

>> x + y
ans =
    1.2434
    0.7045
    1.9281

>> 2 * x
ans =
    0.5570
    1.0938
    1.9150

>> 2 + x
ans =
    2.2785
    2.5469
    2.9575

These all work. But

>> x * y
Error using  * 
Incorrect dimensions for matrix multiplication.

You must use .* for this, but scalar + vector is apparently fine.

So vector + vector and scalar * vector works in Julia, because they work in linear algebra. They also work with .+ and .*, but that’s beside the point. Matlab, on the other hand, allows scalar + vector, but not vector * vector. I don’t see how that is clearer.

Yes, if you use broadcasting:

julia> x = rand(3); y = rand(3);

julia> x .+ y'
3×3 Matrix{Float64}:
 0.906503  1.37568  1.38545
 0.726524  1.1957   1.20547
 1.01696   1.48613  1.49591

Matlab also allows this, as you say, but without the .

1 Like

There are some additional answers here you might find useful.

2 Likes