Possible compiler issue with enzyme and custom data type

I had some tests for a custom data type that were working last Friday. Now, one test doesn’t work, and I’m very confused about what changed. I suspect that there might be an issue with Enzyme and the compiler.

I don’t have a MWE, but in the meantime, here’s a sketch of what I’m doing and the error. I am testing vector products (methods defined elsewhere). Below, a, b, and c are custom types that have dot and cross functions associated with them. Additionally, I’ve defined this dotcross function to test the vector-valued cross function.

function dotcross(x, y, z)
    dot(cross(x, y), z)
end
    @testset "Vector products" begin
        # a·b
        da, db = autodiff(Reverse, dot, Active, Active(a), Active(b))[1]
        @test da.coeffs ==
              ComponentVector{3,Float64,coeff_labels}(sum(a.basis' * b, dims = 2))
        @test db.coeffs ==
              ComponentVector{3,Float64,coeff_labels}(sum(b.basis' * a, dims = 2))

        # (a×b)·c
        da, db, dc = autodiff(Reverse, dotcross, Active, Active(a), Active(b), Active(c))[1]
        @test da.coeffs ≈ ComponentVector{3,Float64,coeff_labels}([
            dot(cross(d1, b), c),
            dot(cross(d2, b), c),
            dot(cross(d3, b), c),
        ])
    end

All of the test above ran correctly last Friday. Now, I get the error below, where enzyme_tests.jl:79 corresponds to the line da, db, dc = autodiff(Reverse, dotcross, Active, Active(a), Active(b), Active(c))[1] above.

  Got exception outside of a @test
  InexactError: check_top_bit(UInt32, -1)
  Stacktrace:
    [1] throw_inexacterror(f::Symbol, ::Type{UInt32}, val::Int32)
      @ Core ./boot.jl:634
    [2] check_top_bit
      @ Core ./boot.jl:648 [inlined]
    [3] toUInt32
      @ Core ./boot.jl:747 [inlined]
    [4] UInt32
      @ Core ./boot.jl:788 [inlined]
    [5] Integer
      @ CEnum ~/.julia/packages/CEnum/B6ahP/src/CEnum.jl:19 [inlined]
    [6] cconvert
      @ CEnum ~/.julia/packages/CEnum/B6ahP/src/CEnum.jl:20 [inlined]
    [7] LLVMAddCallSiteAttribute(C::LLVM.CallInst, Idx::LLVM.API.__JL_Ctag_85, A::LLVM.EnumAttribute)
      @ LLVM.API ~/.julia/packages/LLVM/ycL7C/lib/15/libLLVM.jl:2536
    [8] annotate!(mod::LLVM.Module, mode::Enzyme.API.CDerivativeMode)
      @ Enzyme.Compiler ~/.julia/packages/Enzyme/KJgKj/src/compiler.jl:2709
    [9] codegen(output::Symbol, job::GPUCompiler.CompilerJob{Enzyme.Compiler.EnzymeTarget, Enzyme.Compiler.EnzymeCompilerParams}; libraries::Bool, deferred_codegen::Bool, optimize::Bool, toplevel::Bool, strip::Bool, validate::Bool, only_entry::Bool, parent_job::Nothing)
      @ Enzyme.Compiler ~/.julia/packages/Enzyme/KJgKj/src/compiler.jl:4720
   [10] codegen
      @ ~/.julia/packages/Enzyme/KJgKj/src/compiler.jl:4348 [inlined]
   [11] _thunk(job::GPUCompiler.CompilerJob{Enzyme.Compiler.EnzymeTarget, Enzyme.Compiler.EnzymeCompilerParams}, postopt::Bool) (repeats 2 times)
      @ Enzyme.Compiler ~/.julia/packages/Enzyme/KJgKj/src/compiler.jl:5362
   [12] cached_compilation
      @ ~/.julia/packages/Enzyme/KJgKj/src/compiler.jl:5396 [inlined]
   [13] (::Enzyme.Compiler.var"#509#510"{DataType, UnionAll, DataType, Enzyme.API.CDerivativeMode, NTuple{4, Bool}, Int64, Bool, Bool, UInt64, DataType})(ctx::LLVM.Context)
      @ Enzyme.Compiler ~/.julia/packages/Enzyme/KJgKj/src/compiler.jl:5462
   [14] JuliaContext(f::Enzyme.Compiler.var"#509#510"{DataType, UnionAll, DataType, Enzyme.API.CDerivativeMode, NTuple{4, Bool}, Int64, Bool, Bool, UInt64, DataType})
      @ GPUCompiler ~/.julia/packages/GPUCompiler/U36Ed/src/driver.jl:47
   [15] #s1056#508
      @ ~/.julia/packages/Enzyme/KJgKj/src/compiler.jl:5414 [inlined]
   [16] var"#s1056#508"(FA::Any, A::Any, TT::Any, Mode::Any, ModifiedBetween::Any, width::Any, ReturnPrimal::Any, ShadowInit::Any, World::Any, ABI::Any, ::Any, ::Type, ::Type, ::Type, tt::Any, ::Type, ::Type, ::Type, ::Type, ::Type, ::Any)
      @ Enzyme.Compiler ./none:0
   [17] (::Core.GeneratedFunctionStub)(::UInt64, ::LineNumberNode, ::Any, ::Vararg{Any})
      @ Core ./boot.jl:602
   [18] autodiff(::ReverseMode{false, FFIABI}, ::Const{typeof(dotcross)}, ::Type{Active}, ::Active{CurvilinearVector{3, Float64, 9, (:a, :b, :c), (:∇x, :∇y, :∇z), (:x, :y, :z), Contravariant}}, ::Vararg{Active{CurvilinearVector{3, Float64, 9, (:a, :b, :c), (:∇x, :∇y, :∇z), (:x, :y, :z), Contravariant}}})
      @ Enzyme ~/.julia/packages/Enzyme/KJgKj/src/Enzyme.jl:209
   [19] autodiff(::ReverseMode{false, FFIABI}, ::typeof(dotcross), ::Type, ::Active{CurvilinearVector{3, Float64, 9, (:a, :b, :c), (:∇x, :∇y, :∇z), (:x, :y, :z), Contravariant}}, ::Vararg{Active{CurvilinearVector{3, Float64, 9, (:a, :b, :c), (:∇x, :∇y, :∇z), (:x, :y, :z), Contravariant}}})
      @ Enzyme ~/.julia/packages/Enzyme/KJgKj/src/Enzyme.jl:224
   [20] macro expansion
      @ ~/.julia/dev/PlasmaEquilibriumToolkit/test/enzyme_tests.jl:79 [inlined]
   [21] macro expansion
      @ /usr/share/julia/stdlib/v1.10/Test/src/Test.jl:1577 [inlined]
   [22] macro expansion
      @ ~/.julia/dev/PlasmaEquilibriumToolkit/test/enzyme_tests.jl:72 [inlined]
   [23] macro expansion
      @ /usr/share/julia/stdlib/v1.10/Test/src/Test.jl:1577 [inlined]
   [24] top-level scope
      @ ~/.julia/dev/PlasmaEquilibriumToolkit/test/enzyme_tests.jl:52
   [25] include(fname::String)
      @ Base.MainInclude ./client.jl:489
   [26] top-level scope
      @ ~/.julia/dev/PlasmaEquilibriumToolkit/test/runtests.jl:14
   [27] include(fname::String)
      @ Base.MainInclude ./client.jl:489
   [28] top-level scope
      @ none:6
   [29] eval
      @ Core ./boot.jl:385 [inlined]
   [30] exec_options(opts::Base.JLOptions)
      @ Base ./client.jl:291
   [31] _start()
      @ Base ./client.jl:552
Test Summary:     | Pass  Error  Total   Time
Enzyme tests      |    4      1      5  11.3s
  Sums            |    2             2   8.3s
  Vector products |    2      1      3   3.0s
ERROR: LoadError: Some tests did not pass: 4 passed, 0 failed, 1 errored, 0 broken.

Any idea what I could look for while I cook up a MWE?

There was a chance propagated from a recent version of LLVM.jl, which breaks Enzyme. We’ve since fixed it on Enzyme main, which we hope to release shortly.

Try Enzyme#main in the interim?

I saw your post, but wasn’t sure if it was the same thing.

I’ll try the main branch. Thanks!

Trying Enzyme#main gives the error

ERROR: can not merge projects

when running test, which seems to be an issue with Pkg.jl.

I couldn’t find a workaround. What’s the ETA on getting your fix on Enzyme#main into a release?

Oddly enough, everything works when running with the code with TestEnv.jl.

Released

Everything works now. Thanks!