# Prevent array re-allocation inside a loop (I am not allowed to allocate before the loop)

**URL:** <https://discourse.julialang.org/t/prevent-array-re-allocation-inside-a-loop-i-am-not-allowed-to-allocate-before-the-loop/103042>\
**Category:** Performance\
**Tags:** performance\
**Created:** [August 21, 2023, 5:56pm UTC](https://discourse.julialang.org/t/prevent-array-re-allocation-inside-a-loop-i-am-not-allowed-to-allocate-before-the-loop/103042 "2023-08-21T17:56:17Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![simon79](https://avatars.discourse-cdn.com/v4/letter/s/76d3ee/32.png) [@simon79](https://discourse.julialang.org/u/simon79)\
**Post date:** [August 21, 2023, 5:56pm UTC](https://discourse.julialang.org/t/prevent-array-re-allocation-inside-a-loop-i-am-not-allowed-to-allocate-before-the-loop/103042/1 "2023-08-21T17:56:18Z")

</div>

Hi,  
I am executing a code similar to the simple one below under the constraint that I cannot allocate anything before the loop due to some constraints of the calling function.

```julia
for it=1:ntime
     my_fun()
end

```

where

```julia
my_fun()    
    F = zeros(T, npoin, nelem)
    ...

```

As you can see, this reallocates memory at every loop iteration. I would like to avoid this and only allocate it the first time and reset the array to zero the following times.

The problem would not exist if I could allocate `F` outside of the loop, but I cannot do that.

Is the a way to achieve this?  
thanks

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [August 21, 2023, 6:01pm UTC](https://discourse.julialang.org/t/prevent-array-re-allocation-inside-a-loop-i-am-not-allowed-to-allocate-before-the-loop/103042/2 "2023-08-21T18:01:25Z")

</div>

Can you modify `my_fun()`?

If so, even if not very good for practice, you can define in the global scope:

```julia
const global_F = zeros(T, npoin, nelem)

```

and then inside `my_fun()` do:

```julia
function my_fun()
    F = fill!(global_F, zero(T))

```

but that comes with a lot of possible issues to care about (maintainability, races, etc.).

---

<div class="post-metadata">

**Author:** ![simon79](https://avatars.discourse-cdn.com/v4/letter/s/76d3ee/32.png) [@simon79](https://discourse.julialang.org/u/simon79)\
**Post date:** [August 21, 2023, 6:03pm UTC](https://discourse.julialang.org/t/prevent-array-re-allocation-inside-a-loop-i-am-not-allowed-to-allocate-before-the-loop/103042/3 "2023-08-21T18:03:48Z")

</div>

I can modify my\_fun() and everything else before it’s called, but nothing out of the loop.  
I can try this.  
Feel free to add more options if anything else comes to mind. I don’t particularly like the const option of a global variable.

thanks @lmiq

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [August 21, 2023, 6:11pm UTC](https://discourse.julialang.org/t/prevent-array-re-allocation-inside-a-loop-i-am-not-allowed-to-allocate-before-the-loop/103042/4 "2023-08-21T18:11:23Z")

</div>

> [@simon79](#):
>
> but nothing out of the loop.

Can you explain a little bit this restriction?

(it would prevent even the definition of a global buffer, as I suggested - and complicates things quite a bit, since the variables created in each loop iteration are local to that iteration, so they woudn’t survive for the next iteration to be reused).

One alternative, make `my_fun` a callable struct that carries its buffer:

```julia
struct MyFun{T}
    buffer::Matrix{T}
end

function (my_fun::MyFun)(args)
    F = fill!(my_fun.buffer, 0.0)
    ...
end

const my_fun = MyFun()

```

That is a little bit more clean that the global buffer, as at least the buffer is carried with the struct, but still has to be declared in the global scope (otherwise if you instantiate a new `MyFun` object in each iteration of the loop, the buffer will be recreated the same).

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [August 21, 2023, 6:18pm UTC](https://discourse.julialang.org/t/prevent-array-re-allocation-inside-a-loop-i-am-not-allowed-to-allocate-before-the-loop/103042/5 "2023-08-21T18:18:28Z")

</div>

```julia
for it=1:ntime
    If it == 1
        X = ... # allocate
    end
    my_fun(X)
end

```

?

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [August 21, 2023, 6:20pm UTC](https://discourse.julialang.org/t/prevent-array-re-allocation-inside-a-loop-i-am-not-allowed-to-allocate-before-the-loop/103042/6 "2023-08-21T18:20:18Z")

</div>

Doesn’t work, the `x` is local to the loop iteration:

```julia
julia> function f()
           for i in 1:2
               if i == 1
                   x = zeros(2)
               end
               if i == 2
                   println(" iteration 2 ")
                   x .= 0.0
               end
           end
           return x
       end
f (generic function with 1 method)

julia> f()
 iteration 2 
ERROR: UndefVarError: `x` not defined
Stacktrace:
 [1] f()
   @ Main ./REPL[3]:8
 [2] top-level scope
   @ REPL[4]:1

```

One would need to add something like `local x` outside the loop, but that’s forbidden by the rules 🙂 .

---

<div class="post-metadata">

**Author:** ![GunnarFarneback](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gunnarfarneback/32/1827_2.png) [@GunnarFarneback](https://discourse.julialang.org/u/GunnarFarneback)\
**Post date:** [August 21, 2023, 6:36pm UTC](https://discourse.julialang.org/t/prevent-array-re-allocation-inside-a-loop-i-am-not-allowed-to-allocate-before-the-loop/103042/7 "2023-08-21T18:36:19Z")

</div>

This is somewhat standard:

```julia
let F = zeros(T, npoin, nelem)
    global function my_fun()
        F .= zero(T)
        ...
    end
end

```

but I would rather go for a callable struct.

---

<div class="post-metadata">

**Author:** ![simon79](https://avatars.discourse-cdn.com/v4/letter/s/76d3ee/32.png) [@simon79](https://discourse.julialang.org/u/simon79)\
**Post date:** [August 21, 2023, 7:05pm UTC](https://discourse.julialang.org/t/prevent-array-re-allocation-inside-a-loop-i-am-not-allowed-to-allocate-before-the-loop/103042/8 "2023-08-21T19:05:49Z")

</div>

@Imiq, the issue with the calling function is the following:  
I am using DifferentialEquations.jl to solve `du/dt = RHS`. This packge is used as defined below, where `u` is the solution array and `rhs!` is the function that calculates the RHS vector at every time step.

`function rhs!()` is the only function that I can control when it comes to allocations and array definitions etc. So, if `rhs!()` allocates anything, it does so at every time iteration  
of `solve`.

```julia

 prob = ODEProblem(rhs!,
                      u,
                      tspan,
                      params);
    
solution = solve(prob, solver, dt=Δt...);

```

I cannot pass other than `function rhs!(), u, tspan, params` to `ODEProblem()`.

---

<div class="post-metadata">

**Author:** ![Salmon](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/salmon/32/22968_2.png) [@Salmon](https://discourse.julialang.org/u/Salmon)\
**Post date:** [August 21, 2023, 7:09pm UTC](https://discourse.julialang.org/t/prevent-array-re-allocation-inside-a-loop-i-am-not-allowed-to-allocate-before-the-loop/103042/9 "2023-08-21T19:09:11Z")

</div>

You can preallocate everything in Params.  
Note that you can provide any type you like there so an array or a tuple or your own struct are all fine

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [August 21, 2023, 7:10pm UTC](https://discourse.julialang.org/t/prevent-array-re-allocation-inside-a-loop-i-am-not-allowed-to-allocate-before-the-loop/103042/10 "2023-08-21T19:10:10Z")

</div>

> [@simon79](#):
>
> `function rhs!()` is the only function that I can control when it comes to allocations and array definitions etc. So, if `rhs!()` allocates anything, it does so at every time iteration  
> of `solve`.

You can pass in pre-allocated arrays via `params` etcetera.

See also the [PreallocationTools.jl](https://github.com/SciML/PreallocationTools.jl) package, which is specifically designed for use with `ODEProblem` and similar.

---

<div class="post-metadata">

**Author:** ![simon79](https://avatars.discourse-cdn.com/v4/letter/s/76d3ee/32.png) [@simon79](https://discourse.julialang.org/u/simon79)\
**Post date:** [August 21, 2023, 7:11pm UTC](https://discourse.julialang.org/t/prevent-array-re-allocation-inside-a-loop-i-am-not-allowed-to-allocate-before-the-loop/103042/11 "2023-08-21T19:11:36Z")

</div>

Thank you for clarifying this. PreallocationTools.jl does exactly what I need!