# Help understanding when .= allocates

**URL:** <https://discourse.julialang.org/t/help-understanding-when-allocates/87009>\
**Category:** General Usage\
**Tags:** memory-allocation\
**Created:** [September 9, 2022, 6:28pm UTC](https://discourse.julialang.org/t/help-understanding-when-allocates/87009 "2022-09-09T18:28:31Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![bmit](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bmit/32/12443_2.png) [@bmit](https://discourse.julialang.org/u/bmit)\
**Post date:** [September 9, 2022, 6:28pm UTC](https://discourse.julialang.org/t/help-understanding-when-allocates/87009/1 "2022-09-09T18:28:31Z")

</div>

I could use help understanding the magic of allocations in Julia with the inplace `.=` operation. Below I have 4 ways to write the same operation, but allocations are different for each. Only the first doesn’t allocate. My intention is to preallocate an array and put the output of the `fn` function into that memory. Why are the 2nd, 3rd, and 4th functions allocating when the first isn’t?

```julia
using BenchmarkTools

N, M, L = 3, 5, 4

b = rand(N, M)
c = [rand(N) for _ in 1:L]

a = Array{Float64,3}(undef, N, M, L)

function foo1!(a, b, c)
    for i in eachindex(c)
            a[:, :, i] .= b .- c[i]
    end
    return nothing
end

function foo2!(a, b, c, fn=(-))
    for i in eachindex(c)
            a[:, :, i] .= fn.(b, c[i])
    end
    return nothing
end

function foo3!(a, b, c)
    for i in eachindex(c)
        for j in axes(b, 2)
            a[:, j, i] .= view(b, :, j) - c[i]
        end
    end
    return nothing
end

function foo4!(a, b, c, fn=(-))
    for i in eachindex(c)
        for j in axes(b, 2)
            a[:, j, i] .= fn(view(b, :, j), c[i])
        end
    end
    return nothing
end

@btime foo1!($a, $b, $c) # 71.783 ns (0 allocations: 0 bytes)
@btime foo2!($a, $b, $c) # 1.204 μs (12 allocations: 512 bytes)
@btime foo3!($a, $b, $c) # 864.382 ns (20 allocations: 1.56 KiB)
@btime foo4!($a, $b, $c) # 867.788 ns (20 allocations: 1.56 KiB)

```

---

<div class="post-metadata">

**Author:** ![bgroenks](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bgroenks/32/21784_2.png) [@bgroenks](https://discourse.julialang.org/u/bgroenks)\
**Post date:** [September 9, 2022, 7:02pm UTC](https://discourse.julialang.org/t/help-understanding-when-allocates/87009/2 "2022-09-09T19:02:04Z")

</div>

You are missing broadcast `.`s in functions 3 and 4. Fixing this alone will eliminate allocations in `foo3!`. For the other two functions, you need to add a type argument for `fn` to tell the compiler to specialize on that argument (Julia does not specialize on function arguments by default).

See the updated code below.

```julia
function foo2!(a, b, c, fn::F=(-)) where F
    for i in eachindex(c)
            a[:, :, i] .= fn.(b, c[i])
    end
    return nothing
end

function foo3!(a, b, c)
    for i in eachindex(c)
        for j in axes(b, 2)
            a[:, j, i] .= view(b, :, j) .- c[i]
        end
    end
    return nothing
end

function foo4!(a, b, c, fn::F=(-)) where F
    for i in eachindex(c)
        for j in axes(b, 2)
            a[:, j, i] .= fn.(view(b, :, j), c[i])
        end
    end
    return nothing
end

@btime foo1!($a, $b, $c) # 100.447 ns (0 allocations: 0 bytes)
@btime foo2!($a, $b, $c) # 95.607 ns (0 allocations: 0 bytes)
@btime foo3!($a, $b, $c) # 155.782 ns (0 allocations: 0 bytes)
@btime foo4!($a, $b, $c) # 167.668 ns (0 allocations: 0 bytes)

```

Edit: In general, `.=` will allocate whenever something on the right hand side forces the compiler to allocate an intermediate array. In this case, besides the missing `.`s, it was the lack of type information for `fn` which presumably prevented optimization (and inlining) of the broadcast kernel `fn`.

---

<div class="post-metadata">

**Author:** ![bmit](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bmit/32/12443_2.png) [@bmit](https://discourse.julialang.org/u/bmit)\
**Post date:** [September 9, 2022, 9:02pm UTC](https://discourse.julialang.org/t/help-understanding-when-allocates/87009/3 "2022-09-09T21:02:11Z")

</div>

Thanks! This was super helpful! It doesn’t totally solve my issue though because of another source of confusion on my part.

I was completely misunderstanding what was happening in the first example. I thought the broadcast `.-` between a matrix and a vector was broadcast across the columns of matrix `b` using vector subtraction as in `vec1 - vec2` - similar to how `.-` between a vector and a scalar works. Instead, I see it all just happens at the scalar level.

I’ll pose a different question then. Assuming `fn` returns a vector, it looks like `foo1` and `foo2` simply won’t work. For `foo4` is there any way to skip the allocation on the right hand side of `.=` and get the compiler to write directly into the `a` array?

```julia
function foo4!(a, b, c, fn::F=(-)) where F
    for i in eachindex(c)
        for j in axes(b, 2)
            a[:, j, i] .= fn(view(b, :, j), c[i])
        end
    end
    return nothing
end

```

---

<div class="post-metadata">

**Author:** ![jebej](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jebej/32/1784_2.png) [@jebej](https://discourse.julialang.org/u/jebej)\
**Post date:** [September 9, 2022, 9:06pm UTC](https://discourse.julialang.org/t/help-understanding-when-allocates/87009/4 "2022-09-09T21:06:19Z")

</div>

It looks like `fn` is just `-`, which does not return a vector. Maybe change your example so it’s clearer? Also did you mean the `a` array?

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [September 9, 2022, 9:19pm UTC](https://discourse.julialang.org/t/help-understanding-when-allocates/87009/5 "2022-09-09T21:19:36Z")

</div>

> [@bmit](#):
>
> Assuming `fn` returns a vector, it looks like `foo1` and `foo2` simply won’t work.

For broadcasting to work without allocations `fn` must be a function that maps scalars to scalars. If `fn` inherently maps vectors to vectors then I don’t see how you can avoid allocations.

---

<div class="post-metadata">

**Author:** ![bmit](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bmit/32/12443_2.png) [@bmit](https://discourse.julialang.org/u/bmit)\
**Post date:** [September 9, 2022, 9:27pm UTC](https://discourse.julialang.org/t/help-understanding-when-allocates/87009/6 "2022-09-09T21:27:43Z")

</div>

I apologize for the confusion. I fixed the typo above and will clarify that I mean this method for subtraction.

> -(A::AbstractArray, B::AbstractArray) in Base at arraymath.jl:6

I will also admit I’m not the least bit aware of the capabilities of the Julia compiler, so I have no idea if it’s capable enough to recognize the redundancy of having to create a vector and then copy it.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [September 9, 2022, 9:32pm UTC](https://discourse.julialang.org/t/help-understanding-when-allocates/87009/7 "2022-09-09T21:32:47Z")

</div>

> [@bmit](#):
>
> I will also admit I’m not the least bit aware of the capabilities of the Julia compiler, so I have no idea if it’s capable enough to recognize the redundancy of having to create a vector and then copy it.

See also [Why does Julia need dots to fuse the loops?](https://julialang.org/blog/2017/01/moredots/#why_does_julia_need_dots_to_fuse_the_loops).

---

<div class="post-metadata">

**Author:** ![bmit](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bmit/32/12443_2.png) [@bmit](https://discourse.julialang.org/u/bmit)\
**Post date:** [September 9, 2022, 11:30pm UTC](https://discourse.julialang.org/t/help-understanding-when-allocates/87009/8 "2022-09-09T23:30:17Z")

</div>

Thank you for the reference. It did help me understand what was going on with the first two cases.

Regarding the last two, I think my question then is not necessarily about vectorization. Instead, it’s if there’s a way to tell the compiler, “hey, here’s some memory, just put the output there and error if you can’t.” Originally, I naively thought that’s what `.=` did.

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [September 10, 2022, 8:12am UTC](https://discourse.julialang.org/t/help-understanding-when-allocates/87009/9 "2022-09-10T08:12:06Z")

</div>

> [@bmit](#):
>
> `-(A::AbstractArray, B::AbstractArray)`

If this method is called, it will allocate a new array. It knows nothing about your output array and will create its own.

The way to avoid allocations is to get Julia to call `-(::Float64, ::Float64)` on each element instead, since that just produces a scalar which can be put straight into your output array with no intermediate array allocation. That’s what broadcasting does for you (well, simplified.)

Alternatively, you can write your own function, `minus!(a, b, c)`, which writes straight into `a`, but that would probably just look like this:

```julia
minus!(a, b, c) = (a .= b .- c) 

```

just hiding the same code one level down.

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [September 10, 2022, 8:47am UTC](https://discourse.julialang.org/t/help-understanding-when-allocates/87009/10 "2022-09-10T08:47:49Z")

</div>

> [@bmit](#):
>
> Instead, it’s if there’s a way to tell the compiler, “hey, here’s some memory, just put the output there and error if you can’t.” Originally, I naively thought that’s what `.=` did.

That’s what it does - the allocations in your original example happen before `.=` comes into play though.

Assignment in julia should never allocate, not even broadcasted assignment (unless there’s a custom type that implements `copyto!` in an allocating way, which I’d consider buggy behavior).