Is it possible to add reliable tests that functions do not allocate?

General Usage Performance
,
1

I’m working on a project involving 3D graphics. In this sort of domain, you create a lot of short-term vector and matrix data of a constant size (usually 2D, 3D, and 4D). As well as quaternions, which are stored like a 4D vector.

In this domain, it’s imperative that all this data (and the functions manipulating them) is kept on the stack and type-stable, as the hottest code involves thousands to millions of these small objects. So I’ve been trying to add some unit test functionality that checks whether an expression allocates.

However, in practice it’s extremely fickle. Stuff fails the test that really seems like it shouldn’t; I go in the REPL and try to investigate it myself and find no allocations. I ran Julia with --track-allocation=user, and it tells me that my function’s signature is allocating a bunch of memory.

Is there something wrong with how I’m writing the test? This is my macro (with some detail stripped out):

@macro test_no_allocations(expr)
    return quote
        # Wrap the expression in a function.
        @noinline function run_with_timer()
            return @timed($(esc(expr)))
        end
        # Wrap the test itself in a function, just for good measure.
        @noinline function make_test()
            # Run once to precompile, then test it.
            run_with_timer()
            result = run_with_timer()
            if result.bytes != 0
                error("Test failed, '", $(string(expr)), "' allocated ", result.bytes, " bytes")
            end
        end
        make_test()
    end
end
2

Maybe the allocations come from when the function is compiled? If you use @allocated after having called the function one time first to compile it, do the allocations go away?

3

I am not really answering your question, but these two references might be of use: allocation tests in the CI of a couple of packages:

QuantumClifford.jl: QuantumClifford.jl/test_allocations.jl at master · Krastanov/QuantumClifford.jl · GitHub

Polyester.jl: Polyester.jl/runtests.jl at master · JuliaSIMD/Polyester.jl · GitHub

4

@allocated tends to report some allocations even if there actually are none in the function. You can try running the function multiple times to overcome this: @test @allocated(for _ in 1:1000 f(x) end) < 100.
More package examples:

5