# Allocations when overwriting array elements in-place

**URL:** <https://discourse.julialang.org/t/allocations-when-overwriting-array-elements-in-place/77215>\
**Category:** Performance\
**Tags:** memory-allocation\
**Created:** [March 1, 2022, 1:15am UTC](https://discourse.julialang.org/t/allocations-when-overwriting-array-elements-in-place/77215 "2022-03-01T01:15:58Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![misa](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/misa/32/28917_2.png) [@misa](https://discourse.julialang.org/u/misa)\
**Post date:** [March 1, 2022, 1:15am UTC](https://discourse.julialang.org/t/allocations-when-overwriting-array-elements-in-place/77215/1 "2022-03-01T01:15:58Z")

</div>

I have a relatively large matrix, where I want to set all values below a certain threshold to zero:

```julia
@time A = rand(1000,1000,1000)
@time bool = A .< 0.5
@time A[bool] .= 0.0

5.653554 seconds (2 allocations: 7.451 GiB, 2.64% gc time)
1.698547 seconds (6 allocations: 119.214 MiB, 20.14% gc time)
5.429685 seconds (5 allocations: 3.725 GiB, 1.63% gc time)

```

I would have expected the final step to allocate essentially no memory since I am overwriting elements of an existing array in-place, however the amount of actually allocated memory equals that of creating an entirely new array containing the overwritten values (Julia 1.7.2).

Interestingly, when I instead use a view, the allocations occur during the creation of the view rather than the overwriting step.

```julia
@time Av = @view A[bool]
@time Av .= 0.0

2.786945 seconds (5 allocations: 3.725 GiB, 9.23% gc time)
1.292666 seconds

```

Could anyone help me understand why these allocations occur and whether there is a way to avoid them?

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [March 1, 2022, 2:01am UTC](https://discourse.julialang.org/t/allocations-when-overwriting-array-elements-in-place/77215/2 "2022-03-01T02:01:49Z")

</div>

Those are the allocations of the broadcast mechanism. Put it in a function and those should go away.

---

<div class="post-metadata">

**Author:** ![misa](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/misa/32/28917_2.png) [@misa](https://discourse.julialang.org/u/misa)\
**Post date:** [March 1, 2022, 2:23am UTC](https://discourse.julialang.org/t/allocations-when-overwriting-array-elements-in-place/77215/3 "2022-03-01T02:23:35Z")

</div>

It doesn’t look like it unfortunately - when in a function, the same amount of memory allocated in the final step:

```julia
function func()
    @time A = rand(1000,1000,1000)
    @time bool = A .< 0.5
    @time A[bool] .= 0.0
end

func()

3.824082 seconds (2 allocations: 7.451 GiB, 2.76% gc time)
2.128477 seconds (4 allocations: 119.213 MiB, 18.11% gc time)
4.128135 seconds (4 allocations: 3.725 GiB, 0.22% gc time)

```

---

<div class="post-metadata">

**Author:** ![rdeits](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rdeits/32/286_2.png) [@rdeits](https://discourse.julialang.org/u/rdeits)\
**Post date:** [March 1, 2022, 3:27am UTC](https://discourse.julialang.org/t/allocations-when-overwriting-array-elements-in-place/77215/4 "2022-03-01T03:27:01Z")

</div>

I’m pretty sure this has to do with how the view of `A` is created from `bool`:

```julia
julia> A = rand(100, 100, 100);

julia> bool = A .< 0.5;

julia> using BenchmarkTools

julia> @btime @view($A[$bool]);
  428.214 μs (4 allocations: 3.81 MiB)

```

(Note that I reduced all the dimensions by a factor of 10 to speed up testing).

Even using views here doesn’t help because creating the view of `A` ends up computing the indices of all the nonzero elements of `bool`, which is where the allocation comes from:

```julia
julia> C = @view A[bool];

julia> C.indices
([4, 5, 6, 9, 10, 11, 12, 14, 16, 17 … 999986, 999988, 999989, 999990, 999991, 999992, 999993, 999996, 999998, 1000000],)

```

If you’re primarily interested in removing the allocations, then that’s really easy: Just write a loop:

```julia
julia> function set_using_mask!(A, bool)
         @assert axes(A) == axes(bool)
         @inbounds for i in eachindex(A)
           if bool[i]
             A[i] = 0
           end
         end
       end
set_using_mask! (generic function with 1 method)

julia> @btime ($A[$bool] .= 0);
  1.089 ms (4 allocations: 3.81 MiB)

julia> @btime set_using_mask!($A, $bool)
  3.799 ms (0 allocations: 0 bytes)

```

I was surprised to find that the loop is actually slower than the allocating broadcast. Clearly there’s some interesting benefit to the way `A[bool] .= 0` computes all the nonzero indices first, then iterates over them. Indeed, we can replicate some of that effect:

```julia
julia> function set_using_mask_allocating!(A, bool)
         @assert axes(A) == axes(bool)
         indices = findall(bool)
         @inbounds for i in indices
           A[i] = 0
         end
       end
set_using_mask_allocating! (generic function with 1 method)

julia> @btime set_using_mask_allocating!($A, $bool)
  2.454 ms (2 allocations: 11.44 MiB)

```

I’m definitely surprised that this is faster.

---

<div class="post-metadata">

**Author:** ![shilu1984](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/shilu1984/32/9512_2.png) [@shilu1984](https://discourse.julialang.org/u/shilu1984)\
**Post date:** [March 1, 2022, 3:39am UTC](https://discourse.julialang.org/t/allocations-when-overwriting-array-elements-in-place/77215/5 "2022-03-01T03:39:36Z")

</div>

I think `A[bool]` creates a large temporary matrix：

```julia
julia> @time map!(x -> x < 0.5 ? 0.0 : x, A, A);
 0.840729 seconds (14.05 k allocations: 823.300 KiB, 0.98% compilation time)

```

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [March 1, 2022, 1:07pm UTC](https://discourse.julialang.org/t/allocations-when-overwriting-array-elements-in-place/77215/6 "2022-03-01T13:07:04Z")

</div>

Looks like a job for `replace!`:

```julia
function foo!(A)
    bool = A .< 0.5
    A[bool] .= 0.0
    return A
end

bar!(A) = replace!(x->(x<0.5 ? zero(x) : x), A);

```

Benchmark:

```julia
1.7.2> @btime foo!(A) setup=(A=rand(100,100,100)) evals=1;
  2.003 ms (8 allocations: 3.92 MiB)

1.7.2> @btime bar!(A) setup=(A=rand(100,100,100)) evals=1;
  276.800 μs (0 allocations: 0 bytes)

```