Pure virtual method called terminate called without an active exception

I occasionally encounter this error on Julia v1.8 while using a code that calls Folds.map to perform a multi-threaded calculation on one node of a SLURM cluster:

pure virtual method called
terminate called without an active exception

signal (6): Aborted
in expression starting at none:0
gsignal at /lib64/libc.so.6 (unknown line)
abort at /lib64/libc.so.6 (unknown line)
__verbose_terminate_handler at /workspace/srcdir/gcc-11.1.0/libstdc++-v3/libsupc++/vterminate.cc:95
__terminate at /workspace/srcdir/gcc-11.1.0/libstdc++-v3/libsupc++/eh_terminate.cc:48
terminate at /workspace/srcdir/gcc-11.1.0/libstdc++-v3/libsupc++/eh_terminate.cc:58
__cxa_pure_virtual at /workspace/srcdir/gcc-11.1.0/libstdc++-v3/libsupc++/pure.cc:50
_ZN4llvm11raw_ostream5writeEPKcm at /scratch/user/julia/julia-1.8/bin/../lib/julia/libLLVM-13jl.so (unknown line)
_ZN4llvm11LLVMContext8diagnoseERKNS_14DiagnosticInfoE at /scratch/user/julia/julia-1.8/bin/../lib/julia/libLLVM-13jl.so (unknown line)
_ZNSt17_Function_handlerIFvRKN4llvm12SMDiagnosticEbRKNS0_9SourceMgrERSt6vectorIPKNS0_6MDNodeESaISA_EEEZNS0_28MachineModuleInfoWrapperPass16doInitializationERNS0_6ModuleEEUlS3_bS6_SD_E_E9_M_invokeERKSt9_Any_dataS3_ObS6_SD_ at /scratch/user/julia/julia-1.8/bin/../lib/julia/libLLVM-13jl.so (unknown line)
_ZN4llvm9MCContext8diagnoseERKNS_12SMDiagnosticE at /scratch/user/julia/julia-1.8/bin/../lib/julia/libLLVM-13jl.so (unknown line)
_ZN12_GLOBAL__N_19AsmParser11DiagHandlerERKN4llvm12SMDiagnosticEPv at /scratch/user/julia/julia-1.8/bin/../lib/julia/libLLVM-13jl.so (unknown line)
_ZNK4llvm9SourceMgr12PrintMessageERNS_11raw_ostreamENS_5SMLocENS0_8DiagKindERKNS_5TwineENS_8ArrayRefINS_7SMRangeEEENS8_INS_7SMFixItEEEb at /scratch/user/julia/julia-1.8/bin/../lib/julia/libLLVM-13jl.so (unknown line)
_ZNK4llvm9SourceMgr12PrintMessageENS_5SMLocENS0_8DiagKindERKNS_5TwineENS_8ArrayRefINS_7SMRangeEEENS6_INS_7SMFixItEEEb at /scratch/user/julia/julia-1.8/bin/../lib/julia/libLLVM-13jl.so (unknown line)
_ZN12_GLOBAL__N_19AsmParser3RunEbb at /scratch/user/julia/julia-1.8/bin/../lib/julia/libLLVM-13jl.so (unknown line)
_ZNK4llvm10AsmPrinter13emitInlineAsmENS_9StringRefERKNS_15MCSubtargetInfoERKNS_15MCTargetOptionsEPKNS_6MDNodeENS_9InlineAsm10AsmDialectE at /scratch/user/julia/julia-1.8/bin/../lib/julia/libLLVM-13jl.so (unknown line)
_ZNK4llvm10AsmPrinter13emitInlineAsmEPKNS_12MachineInstrE at /scratch/user/julia/julia-1.8/bin/../lib/julia/libLLVM-13jl.so (unknown line)
_ZN4llvm10AsmPrinter16emitFunctionBodyEv at /scratch/user/julia/julia-1.8/bin/../lib/julia/libLLVM-13jl.so (unknown line)
_ZN4llvm13X86AsmPrinter20runOnMachineFunctionERNS_15MachineFunctionE at /scratch/user/julia/julia-1.8/bin/../lib/julia/libLLVM-13jl.so (unknown line)
_ZN4llvm19MachineFunctionPass13runOnFunctionERNS_8FunctionE at /scratch/user/julia/julia-1.8/bin/../lib/julia/libLLVM-13jl.so (unknown line)
_ZN4llvm13FPPassManager13runOnFunctionERNS_8FunctionE at /scratch/user/julia/julia-1.8/bin/../lib/julia/libLLVM-13jl.so (unknown line)
_ZN4llvm13FPPassManager11runOnModuleERNS_6ModuleE at /scratch/user/julia/julia-1.8/bin/../lib/julia/libLLVM-13jl.so (unknown line)
_ZN4llvm6legacy15PassManagerImpl3runERNS_6ModuleE at /scratch/user/julia/julia-1.8/bin/../lib/julia/libLLVM-13jl.so (unknown line)
operator() at /cache/build/default-amdci4-3/julialang/julia-release-1-dot-8/src/jitlayers.cpp:528
_ZN4llvm3orc14IRCompileLayer4emitESt10unique_ptrINS0_29MaterializationResponsibilityESt14default_deleteIS3_EENS0_16ThreadSafeModuleE at /scratch/user/julia/julia-1.8/bin/../lib/julia/libLLVM-13jl.so (unknown line)
_ZN4llvm3orc31BasicIRLayerMaterializationUnit11materializeESt10unique_ptrINS0_29MaterializationResponsibilityESt14default_deleteIS3_EE at /scratch/user/julia/julia-1.8/bin/../lib/julia/libLLVM-13jl.so (unknown line)
_ZN4llvm3orc19MaterializationTask3runEv at /scratch/user/julia/julia-1.8/bin/../lib/julia/libLLVM-13jl.so (unknown line)
_ZN4llvm6detail18UniqueFunctionBaseIvJSt10unique_ptrINS_3orc4TaskESt14default_deleteIS4_EEEE8CallImplIPFvS7_EEEvPvRS7_ at /scratch/user/julia/julia-1.8/bin/../lib/julia/libLLVM-13jl.so (unknown line)
_ZN4llvm3orc16ExecutionSession22dispatchOutstandingMUsEv at /scratch/user/julia/julia-1.8/bin/../lib/julia/libLLVM-13jl.so (unknown line)
_ZN4llvm3orc16ExecutionSession17OL_completeLookupESt10unique_ptrINS0_21InProgressLookupStateESt14default_deleteIS3_EESt10shared_ptrINS0_23AsynchronousSymbolQueryEESt8functionIFvRKNS_8DenseMapIPNS0_8JITDylibENS_8DenseSetINS0_15SymbolStringPtrENS_12DenseMapInfoISF_EEEENSG_ISD_EENS_6detail12DenseMapPairISD_SI_EEEEEE at /scratch/user/julia/julia-1.8/bin/../lib/julia/libLLVM-13jl.so (unknown line)
_ZN4llvm3orc25InProgressFullLookupState8completeESt10unique_ptrINS0_21InProgressLookupStateESt14default_deleteIS3_EE at /scratch/user/julia/julia-1.8/bin/../lib/julia/libLLVM-13jl.so (unknown line)
_ZN4llvm3orc16ExecutionSession19OL_applyQueryPhase1ESt10unique_ptrINS0_21InProgressLookupStateESt14default_deleteIS3_EENS_5ErrorE at /scratch/user/julia/julia-1.8/bin/../lib/julia/libLLVM-13jl.so (unknown line)
_ZN4llvm3orc16ExecutionSession6lookupENS0_10LookupKindERKSt6vectorISt4pairIPNS0_8JITDylibENS0_19JITDylibLookupFlagsEESaIS8_EENS0_15SymbolLookupSetENS0_11SymbolStateENS_15unique_functionIFvNS_8ExpectedINS_8DenseMapINS0_15SymbolStringPtrENS_18JITEvaluatedSymbolENS_12DenseMapInfoISI_EENS_6detail12DenseMapPairISI_SJ_EEEEEEEEESt8functionIFvRKNSH_IS6_NS_8DenseSetISI_SL_EENSK_IS6_EENSN_IS6_SV_EEEEEE at /scratch/user/julia/julia-1.8/bin/../lib/julia/libLLVM-13jl.so (unknown line)
_ZN4llvm3orc16ExecutionSession6lookupERKSt6vectorISt4pairIPNS0_8JITDylibENS0_19JITDylibLookupFlagsEESaIS7_EERKNS0_15SymbolLookupSetENS0_10LookupKindENS0_11SymbolStateESt8functionIFvRKNS_8DenseMapIS5_NS_8DenseSetINS0_15SymbolStringPtrENS_12DenseMapInfoISK_EEEENSL_IS5_EENS_6detail12DenseMapPairIS5_SN_EEEEEE at /scratch/user/julia/julia-1.8/bin/../lib/julia/libLLVM-13jl.so (unknown line)
_ZN4llvm3orc16ExecutionSession6lookupERKSt6vectorISt4pairIPNS0_8JITDylibENS0_19JITDylibLookupFlagsEESaIS7_EENS0_15SymbolStringPtrENS0_11SymbolStateE at /scratch/user/julia/julia-1.8/bin/../lib/julia/libLLVM-13jl.so (unknown line)
_ZN4llvm3orc16ExecutionSession6lookupENS_8ArrayRefIPNS0_8JITDylibEEENS0_15SymbolStringPtrENS0_11SymbolStateE at /scratch/user/julia/julia-1.8/bin/../lib/julia/libLLVM-13jl.so (unknown line)
_ZN4llvm3orc16ExecutionSession6lookupENS_8ArrayRefIPNS0_8JITDylibEEENS_9StringRefENS0_11SymbolStateE at /scratch/user/julia/julia-1.8/bin/../lib/julia/libLLVM-13jl.so (unknown line)
addModule at /cache/build/default-amdci4-3/julialang/julia-release-1-dot-8/src/jitlayers.cpp:968
jl_add_to_ee at /cache/build/default-amdci4-3/julialang/julia-release-1-dot-8/src/jitlayers.cpp:1261 [inlined]
jl_add_to_ee at /cache/build/default-amdci4-3/julialang/julia-release-1-dot-8/src/jitlayers.cpp:1305
jl_add_to_ee at /cache/build/default-amdci4-3/julialang/julia-release-1-dot-8/src/jitlayers.cpp:1327 [inlined]
_jl_compile_codeinst at /cache/build/default-amdci4-3/julialang/julia-release-1-dot-8/src/jitlayers.cpp:149
jl_generate_fptr_impl at /cache/build/default-amdci4-3/julialang/julia-release-1-dot-8/src/jitlayers.cpp:332
jl_compile_method_internal at /cache/build/default-amdci4-3/julialang/julia-release-1-dot-8/src/gf.c:2081 [inlined]
jl_compile_method_internal at /cache/build/default-amdci4-3/julialang/julia-release-1-dot-8/src/gf.c:2025
_jl_invoke at /cache/build/default-amdci4-3/julialang/julia-release-1-dot-8/src/gf.c:2359 [inlined]
ijl_apply_generic at /cache/build/default-amdci4-3/julialang/julia-release-1-dot-8/src/gf.c:2549
#handle_message#3 at /cache/build/default-amdci4-3/julialang/julia-release-1-dot-8/usr/share/julia/stdlib/v1.8/Logging/src/ConsoleLogger.jl:134
handle_message##kw at /cache/build/default-amdci4-3/julialang/julia-release-1-dot-8/usr/share/julia/stdlib/v1.8/Logging/src/ConsoleLogger.jl:106
unknown function (ip: 0x1553802a8414)
_jl_invoke at /cache/build/default-amdci4-3/julialang/julia-release-1-dot-8/src/gf.c:2367 [inlined]
ijl_apply_generic at /cache/build/default-amdci4-3/julialang/julia-release-1-dot-8/src/gf.c:2549
macro expansion at ./logging.jl:365 [inlined]
profile_printing_listener at /cache/build/default-amdci4-3/julialang/julia-release-1-dot-8/usr/share/julia/stdlib/v1.8/Profile/src/Profile.jl:44
#3 at ./threadingconstructs.jl:258

Unfortunately I don’t have a MWE, and the error doesn’t happen every time. This also occurs at the end of the code, after it has finished executing. I wonder if anyone might shed some light into what I should look at to create an MWE?

The parts that calls the threaded map looks like

function filter_map_nthreads!(c::Channel, nt::Int, λs::AbstractVector{<:AbstractVector},
        vs::AbstractVector{<:AbstractMatrix}, mr::AbstractVector{<:Integer}, matrixfn!; kw...)

    nblasthreads = BLAS.get_num_threads()
    TMapReturnEltype = Tuple{eltype(λs), eltype(vs)}
    try
        BLAS.set_num_threads(max(1, round(Int, nblasthreads/nt)))
        z = zip(λs, vs, mr)
        if length(z) > 0
            Folds.map(z) do (λm, vm, m)
                AB = take!(c)
                Y = filter_map(λm, vm, AB, m, matrixfn!; kw...)
                put!(c, AB)
                Y
            end::Vector{TMapReturnEltype}
        else
            TMapReturnEltype[]
        end
    finally
        BLAS.set_num_threads(nblasthreads)
    end
end

and I’m using the master branch of MKL.jl internally, if that matters.

2 Likes

I also got this error while running Julia on a cluster with PBS scheduler. I use Julia 1.8.2 and don’t (explicitly at least) use the Folds package. But I am using threaded for loops.

1 Like

Same here when using Threads.@threads only. It doesn’t happen consistently, though.

1 Like

Which Julia version are you using? I haven’t seen this in a while on v1.8.2 and above

Edit: it’s back!

I saw the same message on v1.8.4 while running Julia on a cluster with a PBS scheduler. What’s weird is my calculations run perfectly fine, the error comes at the end.

I got this error today when running exit() on a session after running successfully a multithreaded code, when testing Julia v1.9 RC2.

julia> exit()
pure virtual method called
terminate called without an active exception

[16739] signal (6.-6): Aborted
in expression starting at REPL[4]:1
gsignal at /lib64/libc.so.6 (unknown line)
abort at /lib64/libc.so.6 (unknown line)
__verbose_terminate_handler at /workspace/srcdir/gcc-12.1.0/libstdc++-v3/libsupc++/vterminate.cc:95
__terminate at /workspace/srcdir/gcc-12.1.0/libstdc++-v3/libsupc++/eh_terminate.cc:48
terminate at /workspace/srcdir/gcc-12.1.0/libstdc++-v3/libsupc++/eh_terminate.cc:58
__cxa_pure_virtual at /workspace/srcdir/gcc-12.1.0/libstdc++-v3/libsupc++/pure.cc:50
_ZN4llvm11raw_ostream5writeEPKcm at /rhome/FNORO/local/julia-1.9.0-rc2/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm11LLVMContext8diagnoseERKNS_14DiagnosticInfoE at /rhome/FNORO/local/julia-1.9.0-rc2/bin/../lib/julia/libLLVM-14jl.so (unknown line)

It would be good if someone could reproduce this by starting julia as

julia --check-bounds=yes --bug-report=rr

The error report will automatically be uploaded online, and can then be posted on GitHub as an issue. I routinely see this on a cluster, but won’t be able to run rr over there