# Is it possible to add reliable tests that functions do not allocate?

**URL:** <https://discourse.julialang.org/t/is-it-possible-to-add-reliable-tests-that-functions-do-not-allocate/85320>\
**Category:** Performance\
**Tags:** testing, memory-allocation\
**Created:** [August 5, 2022, 12:34am UTC](https://discourse.julialang.org/t/is-it-possible-to-add-reliable-tests-that-functions-do-not-allocate/85320 "2022-08-05T00:34:46Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![heyx3](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/heyx3/32/29811_2.png) [@heyx3](https://discourse.julialang.org/u/heyx3)\
**Post date:** [August 5, 2022, 12:34am UTC](https://discourse.julialang.org/t/is-it-possible-to-add-reliable-tests-that-functions-do-not-allocate/85320/1 "2022-08-05T00:34:47Z")

</div>

I’m working on a project involving 3D graphics. In this sort of domain, you create a lot of short-term vector and matrix data of a constant size (usually 2D, 3D, and 4D). As well as quaternions, which are stored like a 4D vector.

In this domain, it’s imperative that all this data (and the functions manipulating them) is kept on the stack and type-stable, as the hottest code involves thousands to millions of these small objects. So I’ve been trying to add some unit test functionality that checks whether an expression allocates.

However, in practice it’s extremely fickle. Stuff fails the test that really seems like it shouldn’t; I go in the REPL and try to investigate it myself and find no allocations. I ran Julia with `--track-allocation=user`, and it tells me that my function’s _signature_ is allocating a bunch of memory.

Is there something wrong with how I’m writing the test? This is my macro (with some detail stripped out):

```julia
@macro test_no_allocations(expr)
    return quote
        # Wrap the expression in a function.
        @noinline function run_with_timer()
            return @timed($(esc(expr)))
        end
        # Wrap the test itself in a function, just for good measure.
        @noinline function make_test()
            # Run once to precompile, then test it.
            run_with_timer()
            result = run_with_timer()
            if result.bytes != 0
                error("Test failed, '", $(string(expr)), "' allocated ", result.bytes, " bytes")
            end
        end
        make_test()
    end
end

```

---

<div class="post-metadata">

**Author:** ![baggepinnen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/baggepinnen/32/693_2.png) [@baggepinnen](https://discourse.julialang.org/u/baggepinnen)\
**Post date:** [August 5, 2022, 5:39am UTC](https://discourse.julialang.org/t/is-it-possible-to-add-reliable-tests-that-functions-do-not-allocate/85320/2 "2022-08-05T05:39:58Z")

</div>

Maybe the allocations come from when the function is compiled? If you use `@allocated` after having called the function one time first to compile it, do the allocations go away?

---

<div class="post-metadata">

**Author:** ![Krastanov](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/krastanov/32/6817_2.png) [@Krastanov](https://discourse.julialang.org/u/Krastanov)\
**Post date:** [August 5, 2022, 5:58am UTC](https://discourse.julialang.org/t/is-it-possible-to-add-reliable-tests-that-functions-do-not-allocate/85320/3 "2022-08-05T05:58:31Z")

</div>

I am not really answering your question, but these two references might be of use: allocation tests in the CI of a couple of packages:

QuantumClifford.jl: [QuantumClifford.jl/test\_allocations.jl at master · Krastanov/QuantumClifford.jl · GitHub](https://github.com/Krastanov/QuantumClifford.jl/blob/master/test/test_allocations.jl)

Polyester.jl: [Polyester.jl/runtests.jl at master · JuliaSIMD/Polyester.jl · GitHub](https://github.com/JuliaSIMD/Polyester.jl/blob/master/test/runtests.jl#L348)

---

<div class="post-metadata">

**Author:** ![aplavin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/aplavin/32/222056_2.png) [@aplavin](https://discourse.julialang.org/u/aplavin)\
**Post date:** [August 5, 2022, 7:13am UTC](https://discourse.julialang.org/t/is-it-possible-to-add-reliable-tests-that-functions-do-not-allocate/85320/4 "2022-08-05T07:13:12Z")

</div>

`@allocated` tends to report some allocations even if there actually are none in the function. You can try running the function multiple times to overcome this: `@test @allocated(for _ in 1:1000 f(x) end) < 100`.  
More package examples:

- Accessors.jl use BenchmarkTools [Accessors.jl/perf.jl at master · JuliaObjects/Accessors.jl · GitHub](https://github.com/JuliaObjects/Accessors.jl/blob/master/test/perf.jl)
- ConstructionBase.jl does something more fancy [ConstructionBase.jl/runtests.jl at master · JuliaObjects/ConstructionBase.jl · GitHub](https://github.com/JuliaObjects/ConstructionBase.jl/blob/master/test/runtests.jl#L349-L355)

---

<div class="post-metadata">

**Author:** ![jar1](https://avatars.discourse-cdn.com/v4/letter/j/c0e974/32.png) [@jar1](https://discourse.julialang.org/u/jar1)\
**Post date:** [August 5, 2022, 7:50am UTC](https://discourse.julialang.org/t/is-it-possible-to-add-reliable-tests-that-functions-do-not-allocate/85320/5 "2022-08-05T07:50:59Z")

</div>

> <https://github.com/JuliaLang/julia/issues/34248>
>
> When implementing real-time applications (robotics control, streaming audio-sign…al processing, software-defined radio, etc.) in Julia, the unpredictable timing of heap allocation and garbage collection is a significant hazard. Developers who want to use Julia for such applications have to ensure that all required heap memory allocations happen in advance, and not inside time-critical loops. What would help enormously with learning and practicing such allocation-free time-critical programming is a macro that turns allocations into errors. The main purposes of such a macro would be:
> \* Developers can get compile-time feedback on whether they have succeeded in implementing algorithms in a way that cannot trigger allocations and garbage collection
> \* Users of existing allocation-free methods can see in the source code a compiler-enforced promise that this method will not cause allocations
> 
> This feature suggestion was originally made by @rdeits at the end of his wonderful JuliaCon 2018 talk “JuliaRobotics: Making robots walk with Julia”
> https://youtu.be/dmWQtI3DFFo?t=2263
> (also mentioning @tkoolen ) in response to my question of what tools he could envisage to make allocation-free real-time programming easier in Julia:
> https://youtu.be/dmWQtI3DFFo?t=2160
> @JeffBezanson then asked “Would it be enough for the turn-allocation-into-error feature to be lexically scoped”, and after the speaker agreed, promised “OK, you got it!” (followed by audience applause):
> https://youtu.be/dmWQtI3DFFo?t=2330
> 
> What is the status of this feature?