# How to write a unit test for a conditional macro?

**URL:** <https://discourse.julialang.org/t/how-to-write-a-unit-test-for-a-conditional-macro/110168>\
**Category:** General Usage\
**Created:** [February 13, 2024, 6:06pm UTC](https://discourse.julialang.org/t/how-to-write-a-unit-test-for-a-conditional-macro/110168 "2024-02-13T18:06:40Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![rssdev10](https://avatars.discourse-cdn.com/v4/letter/r/e9a140/32.png) [@rssdev10](https://discourse.julialang.org/u/rssdev10)\
**Post date:** [February 13, 2024, 6:06pm UTC](https://discourse.julialang.org/t/how-to-write-a-unit-test-for-a-conditional-macro/110168/1 "2024-02-13T18:06:40Z")

</div>

Hi, is there any way to invalidate the code of a macro? I have implemented a macro that depends on system environment. Now I’m trying to write a unit test for it. I’m able to delay macro code expansion by placing the first call inside eval/parseall. But I can’t force Julia to recompile the macro for the second state of the `DEBUG_OUTPUT` check.

Minimized code:

```julia
macro zc_get_id(title)
    return haskey(ENV, "DEBUG_OUTPUT") ?
        :(get_debug_id($(esc(title)))) :
        :("")
end

```

Unit test now:

```julia
    @testset "zero cost disabled" begin
        ENV["DEBUG_OUTPUT"] = "log | dump"

        eval(Meta.parseall("""
            id = @zc_get_id "test"
            @test !isempty(id)
        """))
    end

    @testset "zero cost enabled" begin
        delete!(ENV, "DEBUG_OUTPUT")

        eval(Meta.parseall("""
            id = @zc_get_id "test"
            @test isempty(id)
        """))
    end

```

Inside the second `eval`, I have already compiled `@zc_get_id` for the case with the environment variable.

One of the workarounds I see - run a separate julia process for testing each branch of the macro. But not sure if this is a good idea.

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [February 13, 2024, 6:10pm UTC](https://discourse.julialang.org/t/how-to-write-a-unit-test-for-a-conditional-macro/110168/2 "2024-02-13T18:10:21Z")

</div>

(I’ve moved the topic to the General category since this is not a question about julias internals or its design)

As macros run at parse time, i.e. before the `"DEBUG_OUTPUT"` entry is deleted from `ENV`, it will always expand based on what `ENV` contains when the macro is expanded.

In general, I’d advise against making your macro dependent on such fickle global state; consider using a function that returns the expression in question instead and testing whether the returned expression is as you expect it to be, passing the environment variable into the function as an argument. That is much easier to test, while keeping the macro itself free from the global state.

---

<div class="post-metadata">

**Author:** ![rssdev10](https://avatars.discourse-cdn.com/v4/letter/r/e9a140/32.png) [@rssdev10](https://discourse.julialang.org/u/rssdev10)\
**Post date:** [February 13, 2024, 6:26pm UTC](https://discourse.julialang.org/t/how-to-write-a-unit-test-for-a-conditional-macro/110168/3 "2024-02-13T18:26:03Z")

</div>

> As macros run at parse time, i.e. before the `"DEBUG_OUTPUT"` entry is deleted from `ENV`, it will always expand based on what `ENV` contains when the macro is expanded.

Therefore I separated changing of this variable and analysis/compiling with eval/parseall. But even after running parseall for the second time, I’m getting:

```julia
julia> @macroexpand @zc_get_id
:(DebugDataWriter.get_debug_id(""))

```

So, Julia keeps previous code of the macro.

The reason why I decided to create this macro - be able to completely exclude from the prod-code any additional components.

The first variable/function based implementation is looking like:

```julia
    using DebugDataWriter

    # Enable adding trace info with the @info macro
    # Each record contains links to the source code and to the saved data file 
    DebugDataWriter.config().enable_log = true

    # Enable saving dumps of data structures
    DebugDataWriter.config().enable_dump = true

    id = get_debug_id("Some query")
    @debug_output id "some complex structure" ones(2, 3)
    @debug_output id "some complex structure" ones(2, 3) :HTML
    @debug_output id "some complex structure" ones(2, 3) :TXT

    # the lambda is executed when enable_dump == true
    @debug_output id "some data structure as a lambda" begin
        zeros(5, 2)
    end

```

But the idea is to completely remove any additional checks. And this is really work but no idea how to cover it by a unit test.

E.g. a zero cost macro wrapper `@zc_dout` looks like:

```julia
macro zc_dout(debug_id, title, data_func)
    is_debug_output_enabled() || return

    return :(@debug_output($(esc(debug_id)), $(esc(title)), $(esc(data_func))))
end

```

So, no code if `DEBUG_OUTPUT` is not defined in a system environment.

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [February 13, 2024, 6:54pm UTC](https://discourse.julialang.org/t/how-to-write-a-unit-test-for-a-conditional-macro/110168/4 "2024-02-13T18:54:09Z")

</div>

> [@rssdev10](#):
>
> So, Julia keeps previous code of the macro.

I’m not sure what exactly you’re referring to - it works for me:

```julia
julia> macro zc_get_id(title)
           return haskey(ENV, "DEBUG_OUTPUT") ?
               :(get_debug_id($(esc(title)))) :
               :("")
       end
@zc_get_id (macro with 1 method)

julia> @macroexpand @zc_get_id "goo"
""

julia> ENV["DEBUG_OUTPUT"] = "bar"
"bar"

julia> @macroexpand @zc_get_id "goo"
:(Main.get_debug_id("goo"))

```

What could be happening is that the macro runs in a different world age when run through your `eval`/`parse` combo, but again, that’s best avoided (especially when dealing with macros).

> [@rssdev10](#):
>
> The reason why I decided to create this macro - be able to completely exclude from the prod-code any additional components.

Like I said above - if the environment variable is not defined in your production system, that macro should do what you intend it to do. For unit testing purposes, a construct like this (extracting the work the macro does into a function) should work, where you then test the output of that function:

```julia
julia> macro zc_get_id(title)
           macro_func(title)
       end
@zc_get_id (macro with 1 method)

julia> function macro_func(title)
           return haskey(ENV, "DEBUG_OUTPUT") ?
               :(get_debug_id($(esc(title)))) :
               :("")
       end
macro_func (generic function with 1 method)

julia> using Test

julia> @testset "all tests" begin
       @testset "zero cost disabled" begin
           ENV["DEBUG_OUTPUT"] = "log | dump"
           @test macro_func("test") != ""
       end

       @testset "zero cost enabled" begin
           delete!(ENV, "DEBUG_OUTPUT")
           @test macro_func("test") == ""
       end
       end;
Test Summary: | Pass Total Time
all tests | 2 2 0.0s

```

The macro will still run at parse-time just the same:

```julia
julia> @macroexpand @zc_get_id "foo"
""

julia> ENV["DEBUG_OUTPUT"] = "bar"
"bar"

julia> @macroexpand @zc_get_id "foo"
:(Main.get_debug_id("foo"))

```

(The latter invocation is parsed after modifying `ENV`, hence expands to the longer version.)

---

<div class="post-metadata">

**Author:** ![rssdev10](https://avatars.discourse-cdn.com/v4/letter/r/e9a140/32.png) [@rssdev10](https://discourse.julialang.org/u/rssdev10)\
**Post date:** [February 13, 2024, 8:36pm UTC](https://discourse.julialang.org/t/how-to-write-a-unit-test-for-a-conditional-macro/110168/5 "2024-02-13T20:36:48Z")

</div>

Ok, thanks, I found my mistake. Looks when I did a series of experiments, I mixed `""` and `nothing` values. But here I shared the minimized code and lost this error.

At the same time, the following code is not working. I think because of the code of `@zc_get_id` was expanded before the assignment of the ENV.

```julia
@testset "check" begin
    ENV["DEBUG_OUTPUT"] = "log | dump"
    id = @zc_get_id "boo"
    @test !isempty(id)

    delete!(ENV, "DEBUG_OUTPUT")
    id = @zc_get_id "goo"
    @test isempty(id)
end

```

But this works with `eval/parseall`.

Regarding the `macro_func` as a function, this is a good idea. The only problem is that the macro itself is not covered by a unit test. So either `macro_func` and a typical set of test checks, or `eval/parse` with running the macro inside …

---

<div class="post-metadata">

**Author:** ![ffevotte](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ffevotte/32/6587_2.png) [@ffevotte](https://discourse.julialang.org/u/ffevotte)\
**Post date:** [February 13, 2024, 9:34pm UTC](https://discourse.julialang.org/t/how-to-write-a-unit-test-for-a-conditional-macro/110168/6 "2024-02-13T21:34:29Z")

</div>

> [@rssdev10](#):
>
> But I can’t force Julia to recompile the macro

I’d advise you to use the same trick as [`ToggleableAsserts.jl`](https://github.com/MasonProtter/ToggleableAsserts.jl), which I think originates from this discourse thread:

> [@@assert alternatives?](https://discourse.julialang.org/t/assert-alternatives/24775/14):
>
> You can also look at taking advantage of constant propogation to get the behavior you want. asserting() = false #making this a function results in code being invalidated and recompiled when this gets changed macro mayassert(test) esc(:(if $(@\_\_MODULE\_\_).asserting() @assert($test) end)) end f(x) = @mayassert x \< 2 If we use that in a function we can see it gets compile out julia\> @code\_llvm f(2) ; @ REPL[3]:1 within `f' define void @julia\_f\_12238(i64) { top: ret void } we can …

With this technique you explicitly toggle the behavior of your macro by redefining a function, which invalidates all code using the macro and forces a recompilation.

`ToggleableAsserts` show how easily such macros can be tested:

> <https://github.com/MasonProtter/ToggleableAsserts.jl/blob/master/test/runtests.jl>

---

<div class="post-metadata">

**Author:** ![bertschi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bertschi/32/33462_2.png) [@bertschi](https://discourse.julialang.org/u/bertschi)\
**Post date:** [February 13, 2024, 10:01pm UTC](https://discourse.julialang.org/t/how-to-write-a-unit-test-for-a-conditional-macro/110168/7 "2024-02-13T22:01:06Z")

</div>

You can always test the macro function – as a function creating expressions – by using `macroexpand`, i.e.,

```julia
@testset "zero cost disabled" begin
    ENV["DEBUG_OUTPUT"] = "log | dump"
    @test "" != @macroexpand Foo.@zc_get_id "test"
end

@testset "zero cost enabled" begin
    delete!(ENV, "DEBUG_OUTPUT")
    @test "" == @macroexpand Foo.@zc_get_id "test"
end

```

which should be rather similar to testing `macro_func` instead.