# Loop over array of static arrays

**URL:** <https://discourse.julialang.org/t/loop-over-array-of-static-arrays/121769>\
**Category:** Performance\
**Tags:** question, staticarrays\
**Created:** [October 26, 2024, 1:16am UTC](https://discourse.julialang.org/t/loop-over-array-of-static-arrays/121769 "2024-10-26T01:16:20Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![nicolas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nicolas/32/26439_2.png) [@nicolas](https://discourse.julialang.org/u/nicolas)\
**Post date:** [October 26, 2024, 1:16am UTC](https://discourse.julialang.org/t/loop-over-array-of-static-arrays/121769/1 "2024-10-26T01:16:20Z")

</div>

I have the following loop over arrays (the function `double()` will obviously be more complicated, but the number of arrays is always the same)

```julia

double(x) = 2x
function doubles_using_arrays()
    xs = [ones(10) for i in 1:10]
    for i in 1:9
        xs[i+1] .= double(xs[i])
        # Do some more stuff
    end
    xs
end

```

I was wondering whether I could accelerate this using static arrays. A naive way would be to simply replace

```julia
   xs = [ones(10) for i in 1:10] 

```

with

```julia
    xs = [SA[ones(10)] for i in 1:10]

```

and not broadcast. The problems is that since the vectors live on the heap, this offers only mild speed gains. The proper (yet still naive) way to take advantage of static arrays would be:

```julia
function doubles_using_SA()
    x1 = SA[ones(10)]
    # Do some more stuff
    x2 = double(x1)
    # Do some more stuff
    x3 = double(x2)
    # Do some more stuff
    x4 = double(x3)
    # Do some more stuff
    x5 = double(x4)
    # Do some more stuff
    x6 = double(x5)
    # Do some more stuff
    x7 = double(x6)
    # Do some more stuff
    x8 = double(x7)
    # Do some more stuff
    x9 = double(x8)
    # Do some more stuff
    x10 = double(x9)
    # Do some more stuff
    return x1, x2, x3, x4, x5, x6, x7, x8, x9, x10
end

```

Obviously, this gets very long, especially given that `# Do some more stuff` itself contains a few lines. My question is: is there a way to write `doubles_using_SA()` using a loop, where `x1`,…,`x10` live on the stack, but can be referred to using `i` and `i+1`? I imagine this could be done using metaprogramming, but I’ve never used it and given what is said about it, I question whether it is worth the effort and complexity (and could itself add significant computation time).

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [October 26, 2024, 1:32am UTC](https://discourse.julialang.org/t/loop-over-array-of-static-arrays/121769/2 "2024-10-26T01:32:27Z")

</div>

> [@nicolas](#):
>
> I was wondering whether I could accelerate this using static arrays.

Yup. The following is about 7x faster for me:

```julia
function doubles_using_sarrays()
    xs = [@SVector(ones(10)) for i in 1:10]
    for i in 1:9
        xs[i+1] = double(xs[i])
        # Do some more stuff
    end
    xs
end

```

Key points:

1. Don’t call `ones(10)` and _then_ convert to a StaticArray. Not only does this first heap-allocate a `ones(10)` array before converting, but it is also **type-unstable**. The key to efficient use of StaticArrays is to **let the compiler know the length of the array**. The `@SVector` macro rewrites the `ones(10)` into a call to `StaticArrays.ones((SVector){10})`, which puts the length `10` in the type. — this is a common confusion, see e.g. [Optimizing function with vector reassignments - #8 by stevengj](https://discourse.julialang.org/t/optimizing-function-with-vector-reassignments/120938/8)
2. Since `SVector` is immutable, use `=` instead of `.=`.

(The only remaining heap allocation is the container `xs`. You could use a static array for that as well, if you want, but as the size of the object increases the advantage of static arrays is less. And, again, you only get an advantage if you know the length at compile time. Does it really make sense in your application to make the length a constant?)

---

<div class="post-metadata">

**Author:** ![Benny](https://avatars.discourse-cdn.com/v4/letter/b/49beb7/32.png) [@Benny](https://discourse.julialang.org/u/Benny)\
**Post date:** [October 26, 2024, 1:32am UTC](https://discourse.julialang.org/t/loop-over-array-of-static-arrays/121769/3 "2024-10-26T01:32:41Z")

</div>

Putting aside what happens in `# Do some more stuff` for now, things tend to live on heap so references can share them and their changes. If you don’t need that, I think you could work with StaticArrays.jl’s immutables and reinstantiate instead of mutate, likely easier via Accessors.jl. Thing is, I haven’t really tried doing this for any element bigger than a handful of scalars so I don’t know if things can be compiled to fiddling with some bits in a modest stack-allocated array before copying on return. I tend to go for preallocated arrays in addition to smaller elements.

Update, haven’t bothered to dig into the compiled code, but the performance hit suggests that the compiler doesn’t make it as efficient as an allocated vector. At least it shaves the allocations off. `double_using_sarrays` also seems a tad faster on v1.10.5 and with only 1 allocation (no separate `Memory` from `Vector`), and v1.11.1 is sporadically compiled with 20 allocations instead of 2. A recent thread has pointed out this odd allocation.

```julia
julia> function doubles_using_sarrays2()
           xs = @SVector([@SVector(ones(10)) for i in 1:10])
           for i in 1:9
               @reset xs[i+1] = double(xs[i])
               # Do some more stuff
           end
           xs
       end
doubles_using_sarrays2 (generic function with 1 method)

julia> doubles_using_sarrays() == doubles_using_sarrays2()
true

julia> @benchmark doubles_using_sarrays()
BenchmarkTools.Trial: 10000 samples with 958 evaluations.
 Range (min … max): 94.676 ns … 1.385 μs ┊ GC (min … max): 0.00% … 75.39%
 Time (median): 100.522 ns ┊ GC (median): 0.00%
 Time (mean ± σ): 146.475 ns ± 118.044 ns ┊ GC (mean ± σ): 14.76% ± 16.52%

  █▄▃▃▂▃▂▂▂▂▄▃▂▁ ▁▁ ▁
  ██████████████▇▇▇▇█▇▇▅▆▅▄▆▅▅▇████▇▇▆▇▆▅▆▅▅▅▅▅▄▁▄▅▅▄▄▄▃▄▄▄▅▅▄▅ █
  94.7 ns Histogram: log(frequency) by time 731 ns <

 Memory estimate: 928 bytes, allocs estimate: 2.

julia> @benchmark doubles_using_sarrays2()
BenchmarkTools.Trial: 10000 samples with 346 evaluations.
 Range (min … max): 261.561 ns … 1.692 μs ┊ GC (min … max): 0.00% … 0.00%
 Time (median): 274.277 ns ┊ GC (median): 0.00%
 Time (mean ± σ): 332.281 ns ± 136.370 ns ┊ GC (mean ± σ): 0.00% ± 0.00%

  █▇▂▆▂▂▂▁▁ ▁▂▁ ▁▂ ▂ ▁ ▁
  ████████████████▇█▇▇▇███▇█▇██▇██▇███▇▇▇▆▇▆▇▇▆▆▅▅▆▅▄▄▅▄▄▅▅▃▄▄▅ █
  262 ns Histogram: log(frequency) by time 881 ns <

 Memory estimate: 0 bytes, allocs estimate: 0.

```

---

<div class="post-metadata">

**Author:** ![nicolas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nicolas/32/26439_2.png) [@nicolas](https://discourse.julialang.org/u/nicolas)\
**Post date:** [October 26, 2024, 1:54am UTC](https://discourse.julialang.org/t/loop-over-array-of-static-arrays/121769/4 "2024-10-26T01:54:12Z")

</div>

Thanks for the tip. Yes I wasn’t too careful. If I modify my `doubles_using_SA()` the same way, I still get a ∼4X speed gain compared to `doubles_using_sarrays()` though (even making xs a MVector).

---

<div class="post-metadata">

**Author:** ![nicolas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nicolas/32/26439_2.png) [@nicolas](https://discourse.julialang.org/u/nicolas)\
**Post date:** [October 26, 2024, 2:04am UTC](https://discourse.julialang.org/t/loop-over-array-of-static-arrays/121769/5 "2024-10-26T02:04:00Z")

</div>

Thanks. As I understand it, reset creates a modified copy of the object, so it copies the whole xs every time.

For now, I think a feasible solution for me could be to change my algorithm in such a way that I don’t need to keep all arrays:

```julia
    x_old = @SVector(ones(10))
    for i in 1:9
        x_new = double(x_old)
        # Accumulate some more stuff using x_new
        x_old = x_new
    end

```

but I could imagine ways in which this wouldn’t be possible.

---

<div class="post-metadata">

**Author:** ![Benny](https://avatars.discourse-cdn.com/v4/letter/b/49beb7/32.png) [@Benny](https://discourse.julialang.org/u/Benny)\
**Post date:** [October 26, 2024, 7:07am UTC](https://discourse.julialang.org/t/loop-over-array-of-static-arrays/121769/6 "2024-10-26T07:07:00Z")

</div>

> [@nicolas](#):
>
> reset creates a modified copy of the object, so it copies the whole xs every time.

Semantically it’s instantiating another value. Whether that involves overhauls or small changes is up to the compiler figuring out what it can get away with to pull off your code. Hypothetically, a compiler could do `@reset massivestruct.x += 1` with the same efficient change on the stack as `x += 1`. StaticArrays.jl does say that its current implementation stresses the compiler and gets slower than heap-allocated `Array`s at larger sizes, the pre-1.11 rule of thumb being 100 elements like in your example. Silver lining is there’s room for improvement.

It’s also worth pointing out that returning `xs` is a problem even with the most efficient implementation on the stack because it needs to copy that array before the stack frame goes, whereas a heap-allocated array only needs its pointer to be copied around. 800 bytes isn’t too bad once, but occupying multiple spots in the call stack or scaling to larger lengths is a real memory hog where there is much less memory (relative to heap) to go around. For a similar reason to variable-length arrays on the stack going the way of the dinosaurs, Julia is designed to lean toward easily scalable collections on the heap and immutable individual values on the stack, assuming it’s unusual to manually make a massive composite.

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [October 26, 2024, 7:42am UTC](https://discourse.julialang.org/t/loop-over-array-of-static-arrays/121769/7 "2024-10-26T07:42:31Z")

</div>

> [@nicolas](#):
>
> `xs[i+1] .= double(xs[i])`

Before going to StaticArrays, could you try using regular Arrays, but with proper dotting?

```julia
xs[i+1] .= double.(xs[i])

```

---

<div class="post-metadata">

**Author:** ![Benny](https://avatars.discourse-cdn.com/v4/letter/b/49beb7/32.png) [@Benny](https://discourse.julialang.org/u/Benny)\
**Post date:** [October 26, 2024, 7:48am UTC](https://discourse.julialang.org/t/loop-over-array-of-static-arrays/121769/8 "2024-10-26T07:48:19Z")

</div>

To help clarify, that extra dot changes the in-place broadcast kernel, saves the looped allocations done by a scalar-array multiplication in `double(xs[i])`. But that only trims it from 40 (2 per `Array` now) to the 22 before the loop in the example.

---

<div class="post-metadata">

**Author:** ![nicolas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nicolas/32/26439_2.png) [@nicolas](https://discourse.julialang.org/u/nicolas)\
**Post date:** [October 26, 2024, 12:30pm UTC](https://discourse.julialang.org/t/loop-over-array-of-static-arrays/121769/9 "2024-10-26T12:30:38Z")

</div>

Thanks. Yes of course. My algorithm (with vectors) modifies in place:

```julia
function double!(x_out, x_in)
    x_out .= 2x_in
end

```

I was just trying to come up with the simplest self-contained illustration of my question.

---

<div class="post-metadata">

**Author:** ![jling](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jling/32/212909_2.png) [@jling](https://discourse.julialang.org/u/jling)\
**Post date:** [October 26, 2024, 12:43pm UTC](https://discourse.julialang.org/t/loop-over-array-of-static-arrays/121769/10 "2024-10-26T12:43:12Z")

</div>

Why not use ArraysOfArrays.jl?

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [October 26, 2024, 12:48pm UTC](https://discourse.julialang.org/t/loop-over-array-of-static-arrays/121769/11 "2024-10-26T12:48:34Z")

</div>

> [@nicolas](#):
>
> `x_out .= 2x_in`

You have the same problem here, the right-hand-side allocates an intermediate array. Try

```julia
x_out .= 2 .* x_in

```

instead.

> [@nicolas](#):
>
> I was just trying to come up with the simplest self-contained illustration of my question.

I actually don’t think this was nitpicking, btw, your MWE had a fatal flaw that undermined the entire example.

MWEs are, contrary to common opinion, very hard to make, and can be a lot of work.

---

<div class="post-metadata">

**Author:** ![nicolas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nicolas/32/26439_2.png) [@nicolas](https://discourse.julialang.org/u/nicolas)\
**Post date:** [October 26, 2024, 2:29pm UTC](https://discourse.julialang.org/t/loop-over-array-of-static-arrays/121769/12 "2024-10-26T14:29:07Z")

</div>

Thanks for the suggestion. I didn’t know this package. From what I understand, I think it creates normal arrays with views, no matter whether I create them as SArrays or not. But maybe you have something different in mind.

```julia
julia> using ArraysOfArrays, StaticArrays;

julia> VA = VectorOfArrays{Float64, 2}();

julia> push!(VA, ones(2, 3));

julia> push!(VA, @SArray(ones(2, 3)));

julia> typeof(VA[1])
Base.ReshapedArray{Float64, 2, SubArray{Float64, 1, Vector{Float64}, Tuple{UnitRange{Int64}}, true}, Tuple{}}

julia> typeof(VA[2])
Base.ReshapedArray{Float64, 2, SubArray{Float64, 1, Vector{Float64}, Tuple{UnitRange{Int64}}, true}, Tuple{}}

```

---

<div class="post-metadata">

**Author:** ![jling](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jling/32/212909_2.png) [@jling](https://discourse.julialang.org/u/jling)\
**Post date:** [October 26, 2024, 3:29pm UTC](https://discourse.julialang.org/t/loop-over-array-of-static-arrays/121769/13 "2024-10-26T15:29:15Z")

</div>

Correct, but it’s another way to avoid pointer chasing for inner array