# Feedforward NN using StaticArrays with no allocation

**URL:** <https://discourse.julialang.org/t/feedforward-nn-using-staticarrays-with-no-allocation/20036>\
**Category:** Performance\
**Tags:** memory-allocation\
**Created:** [January 24, 2019, 2:20pm UTC](https://discourse.julialang.org/t/feedforward-nn-using-staticarrays-with-no-allocation/20036 "2019-01-24T14:20:07Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![bender9000](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bender9000/32/9464_2.png) [@bender9000](https://discourse.julialang.org/u/bender9000)\
**Post date:** [January 24, 2019, 2:20pm UTC](https://discourse.julialang.org/t/feedforward-nn-using-staticarrays-with-no-allocation/20036/1 "2019-01-24T14:20:07Z")

</div>

I am writing a performance sensitive application that cannot have any allocations which also runs inference using a multi layered perceptron. The dimensions of each layers output in Neural Network are fairly small and benchmarking individual layers forward pass stage revealed that using StaticArrays was faster.

My question is how do I implement the inference step in an elegant way using StaticArrays. My current implementation uses Matrix and Vectors for weights and bias of each layer and A\_mul\_B! to avoid any allocations.

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [January 24, 2019, 2:28pm UTC](https://discourse.julialang.org/t/feedforward-nn-using-staticarrays-with-no-allocation/20036/2 "2019-01-24T14:28:18Z")

</div>

Just use out of place. StaticArrays are stack-allocated structures so they won’t allocate when you create them. They’re more like a high dimensional number, like a complex number or a Float64. They don’t heap-allocate memory. So the algorithm is just `tanh(W2*sigma(W1*x))` etc.

---

<div class="post-metadata">

**Author:** ![bender9000](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bender9000/32/9464_2.png) [@bender9000](https://discourse.julialang.org/u/bender9000)\
**Post date:** [January 24, 2019, 2:44pm UTC](https://discourse.julialang.org/t/feedforward-nn-using-staticarrays-with-no-allocation/20036/3 "2019-01-24T14:44:27Z")

</div>

Thanks for your response. I am still getting allocations when I run this simple foobar example in Julia 0.6. I think I might be missing something.

```julia
using BenchmarkTools
using StaticArrays

@inline relu{T <: AbstractFloat}(x::T) = max(zero(T), x)

function profile()
    input = @SVector rand(5);
    W1 = @SMatrix rand(10,5); b1 = @SVector rand(10);
    W2 = @SMatrix rand(1,10); b2 = @SVector rand(1);
    
    @btime W2*relu.(W1*input+b1)+b2
end

profile()
# prints the following
# 274.584 ns (5 allocations: 320 bytes)
# 1-element StaticArrays.SArray{Tuple{1},Float64,1,1}:
# 11.5083

```

---

<div class="post-metadata">

**Author:** ![bender9000](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bender9000/32/9464_2.png) [@bender9000](https://discourse.julialang.org/u/bender9000)\
**Post date:** [January 24, 2019, 2:49pm UTC](https://discourse.julialang.org/t/feedforward-nn-using-staticarrays-with-no-allocation/20036/4 "2019-01-24T14:49:38Z")

</div>

@ChrisRackauckas Actually, I take that back. If use `@allocated` instead of `@btime` I get zero allocations. Not sure what’s happening there. I am assuming that means there are no allocations in practice.

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [January 24, 2019, 2:49pm UTC](https://discourse.julialang.org/t/feedforward-nn-using-staticarrays-with-no-allocation/20036/5 "2019-01-24T14:49:53Z")

</div>

Is this coded on Julia 0.6? 1.0 has a lot of performance fixes.

Also, interpolate the variables into the `@btime` macro, or make them const.

---

<div class="post-metadata">

**Author:** ![bender9000](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bender9000/32/9464_2.png) [@bender9000](https://discourse.julialang.org/u/bender9000)\
**Post date:** [January 24, 2019, 3:02pm UTC](https://discourse.julialang.org/t/feedforward-nn-using-staticarrays-with-no-allocation/20036/6 "2019-01-24T15:02:20Z")

</div>

Yes, it is coded in Julia 0.6 (edited the comment to state it explicitly). I guess the few allocations were because of the macro creating the closure. Following what you said caused it to have zero allocations as shown below:

```julia
using BenchmarkTools
using StaticArrays

@inline relu{T <: AbstractFloat}(x::T) = max(zero(T), x)

function profile()
    @btime W2*relu.(W1*input+b1)+b2 setup=(input = @SVector rand(5); W1 = @SMatrix rand(10,5); b1 = @SVector rand(10); W2 = @SMatrix rand(1,10); b2 = @SVector rand(1);)
end

profile()
# 22.266 ns (0 allocations: 0 bytes)
# 1-element StaticArrays.SArray{Tuple{1},Float64,1,1}:
# 10.2185
```