# Unit test for constant folding success

**URL:** <https://discourse.julialang.org/t/unit-test-for-constant-folding-success/124625>\
**Category:** General Usage\
**Tags:** testing, constant-propagation, testjl-unittesting\
**Created:** [January 10, 2025, 8:49am UTC](https://discourse.julialang.org/t/unit-test-for-constant-folding-success/124625 "2025-01-10T08:49:05Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![nsajko](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nsajko/32/221187_2.png) [@nsajko](https://discourse.julialang.org/u/nsajko)\
**Post date:** [January 10, 2025, 8:49am UTC](https://discourse.julialang.org/t/unit-test-for-constant-folding-success/124625/1 "2025-01-10T08:49:05Z")

</div>

Here’s a proposal for how to test for successful constant folding:

```julia
using Test

function test_constant_folding(f, r)
    vec = code_typed(f, Tuple{})
    p = only(vec)
    code_info = first(p)
    code = try
        code_info.code
    catch
        nothing
    end
    @test repr(only(code)) == repr(:(return $r)) skip=(code isa Nothing)
end

# some example functions to test for foldability
function f(value = 9)
    it_fil = Iterators.filter((_ -> true), value)
    it_map = Iterators.map((x -> x + 1), it_fil)
    v = collect(it_map)
    only(v)
end
function g(collection = (2, 6))
    it_fil = Iterators.filter((_ -> true), collection)
    it_map = Iterators.map((x -> x + 1), it_fil)
    v = collect(it_map)
    sum(v)
end

@testset "constant folding" begin
    test_constant_folding((() -> 7), 7)
    test_constant_folding(f, 10)
    test_constant_folding(g, 10)
end

```

In the REPL:

```julia-repl
julia> @testset "constant folding" begin
           test_constant_folding((() -> 7), 7)
           test_constant_folding(f, 10)
           test_constant_folding(g, 10)
       end
constant folding: Error During Test at REPL[2]:11
  Test threw exception
  Expression: repr(only(code)) == repr(:(return $r))
  ArgumentError: Collection has multiple elements, must contain exactly 1 element
  Stacktrace:
   [1] _only
     @ ./iterators.jl:1545 [inlined]
   [2] only(x::Vector{Any})
     @ Base.Iterators ./iterators.jl:1536
   [3] macro expansion
     @ ~/tmp/jl/jl/nightly_normal/julia-3d85309e80/share/julia/stdlib/v1.12/Test/src/Test.jl:676 [inlined]
   [4] test_constant_folding(f::Function, r::Int64)
     @ Main ./REPL[2]:11
constant folding: Error During Test at REPL[2]:11
  Test threw exception
  Expression: repr(only(code)) == repr(:(return $r))
  ArgumentError: Collection has multiple elements, must contain exactly 1 element
  Stacktrace:
   [1] _only
     @ ./iterators.jl:1545 [inlined]
   [2] only(x::Vector{Any})
     @ Base.Iterators ./iterators.jl:1536
   [3] macro expansion
     @ ~/tmp/jl/jl/nightly_normal/julia-3d85309e80/share/julia/stdlib/v1.12/Test/src/Test.jl:676 [inlined]
   [4] test_constant_folding(f::Function, r::Int64)
     @ Main ./REPL[2]:11
Test Summary: | Pass Error Total Time
constant folding | 1 2 3 2.1s
RNG of the outermost testset: Random.Xoshiro(0x2bc1148ebb7c1f60, 0x8ecccae6baa3e4fe, 0xc9fc8bc1405b0303, 0xd75d65d894272b1b, 0xada3f8f53a410b26)
ERROR: Some tests did not pass: 1 passed, 0 failed, 2 errored, 0 broken.

```

I think this test is supposed to work as expected as long as `CodeInfo` has an iterator of instructions in its `:code` field. When the property access fails, the test should be skipped gracefully, though.

I wonder if anyone has comments, in particular regarding reliability of the test. Are there any unexpected ways this could fail? cc @Sukera

Regarding possible alternatives, a weaker test would be to check, e.g., with `Base.infer_effects` or `InteractiveUtils.@infer_effects` (neither is public 😅), whether the call infers as `:foldable`. However that doesn’t guarantee that it’ll actually _be_ constant folded, AFAIK. Also, there’s no public API to check whether an `Effects` is `:foldable`, even though we have the non-public `Core.Compiler.is_foldable`.

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [January 10, 2025, 9:01am UTC](https://discourse.julialang.org/t/unit-test-for-constant-folding-success/124625/2 "2025-01-10T09:01:17Z")

</div>

Yes, this can fail in CI due to code coverage metadata. I’ve done a similar implementation in RequiredInterfaces.jl and had to add a bunch of almost arbitrary additional guards just for handling coverage correctly. It’s a very brittle piece of code that breaks often, and I’m not even trying to do this for arbitrary code, so I imagine it would be even worse for your usecase.

---

<div class="post-metadata">

**Author:** ![nsajko](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nsajko/32/221187_2.png) [@nsajko](https://discourse.julialang.org/u/nsajko)\
**Post date:** [January 10, 2025, 9:05am UTC](https://discourse.julialang.org/t/unit-test-for-constant-folding-success/124625/3 "2025-01-10T09:05:10Z")

</div>

> [@Sukera](#):
>
> fail in CI due to code coverage metadata

Ah, yes, I know about that. We sort of work around that in FixedSizeArrays.jl by having a separate _production_ build in CI, without `--check-bounds=yes` or code coverage, so it’s supposed to have good effects.

I was looking more for interfaces the test depends on, but that might break in future Julia versions.

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [January 10, 2025, 9:19am UTC](https://discourse.julialang.org/t/unit-test-for-constant-folding-success/124625/4 "2025-01-10T09:19:16Z")

</div>

> [@nsajko](#):
>
> I was looking more for interfaces the test depends on, but that might break in future Julia versions.

There are none that I know of. As far as I know, effects are the only correct way to do this and they are not public/reliable API yet but rather an internal implementation detail of the compiler. The only other option I can think of is checking whether the compiler internally is able to infer the return “type” as a `Core.Const`, but this is also highly internal, so not a solution for you.

---

<div class="post-metadata">

**Author:** ![nsajko](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nsajko/32/221187_2.png) [@nsajko](https://discourse.julialang.org/u/nsajko)\
**Post date:** [January 10, 2025, 9:45am UTC](https://discourse.julialang.org/t/unit-test-for-constant-folding-success/124625/5 "2025-01-10T09:45:45Z")

</div>

> [@Sukera](#):
>
> As far as I know, effects are the only correct way to do this and they are not public/reliable API yet but rather an internal implementation detail of the compiler.

What’s wrong with my approach above, based on `code_typed`? `code_typed` _is_ public, for what that’s worth. Its interface isn’t completely specified, so my idea is to skip the test if anything unexpected happens.

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [January 10, 2025, 10:52am UTC](https://discourse.julialang.org/t/unit-test-for-constant-folding-success/124625/6 "2025-01-10T10:52:33Z")

</div>

It relies on `repr` being invariant under quote interpolation, which may not be the case. For simple cases like integers and floats this works, because these objects are literals & placed into the IR directly, but for more complicated expressions this is no longer the case:

```julia
julia> struct Foo
           a::Int
       end

julia> test_constant_folding(() -> Foo(1), Foo(1))
Test Failed at REPL[9]:10
  Expression: repr(only(code)) == repr(:(return $r))
   Evaluated: ":(return \$(QuoteNode(Foo(1))))" == ":(return Foo(1))"

ERROR: There was an error during testing

```

Even without `repr` your property doesn’t hold, because interpolation is not in general the same as when a value is constructed by the compiler:

```julia
#=
           dump(only(code))
           cmp = :(return $r)
           dump(cmp)
=#
Core.ReturnNode
  val: QuoteNode
    value: Foo
      a: Int64 1
Expr
  head: Symbol return
  args: Array{Any}((1,))
    1: Foo
      a: Int64 1

```

That’s why I’m saying that the “most blessed” way of doing this is through the constant folding effects/type inference.

---

<div class="post-metadata">

**Author:** ![nsajko](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nsajko/32/221187_2.png) [@nsajko](https://discourse.julialang.org/u/nsajko)\
**Post date:** [January 10, 2025, 11:02am UTC](https://discourse.julialang.org/t/unit-test-for-constant-folding-success/124625/7 "2025-01-10T11:02:52Z")

</div>

True, thank you. I think the above code may still be useful for some test suites, though, as long as the return type in question works for `repr`, e.g., integers and floats, as you say.