OrdinaryDiffEq fails to precompile in Julia 1.9.3

I am suddenly getting an error while precompiling the OrdinaryDiffEq package. I am using Julia on a cluster and it had been working fine until last night. But I tried to run my code using it and I get the following error :


**[ Info:** Precompiling OrdinaryDiffEq [1dea7af3-3e70-54e6-95c3-0bf5283fa5ed]

Invalid instruction at 0x7f31ca5d4b30: 0xc5, 0xfb, 0x92, 0xc8, 0xc5, 0xfb, 0x10, 0x05, 0x44, 0x99, 0xfd, 0xff, 0x62, 0xf1, 0xff

[852143] signal (4.2): Illegal instruction

in expression starting at /home/pvinod_umassd_edu/.julia/packages/OrdinaryDiffEq/EQy59/src/tableaus/rosenbrock_tableaus.jl:136

Float64 at ./float.jl:165 [inlined]

convert at ./number.jl:7 [inlined]

_promote at ./promotion.jl:365 [inlined]

promote at ./promotion.jl:387 [inlined]

gemm_wrapper! at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/LinearAlgebra/src/matmul.jl:667

mul! at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/LinearAlgebra/src/matmul.jl:161 [inlined]

mul! at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/LinearAlgebra/src/matmul.jl:276 [inlined]

* at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/LinearAlgebra/src/matmul.jl:148 [inlined]

_transformtab at /home/pvinod_umassd_edu/.julia/packages/OrdinaryDiffEq/EQy59/src/generic_rosenbrock.jl:841

ROS34PW1aTableau at /home/pvinod_umassd_edu/.julia/packages/OrdinaryDiffEq/EQy59/src/generic_rosenbrock.jl:871

unknown function (ip: 0x7f31d63db4ff)

_jl_invoke at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2758 [inlined]

ijl_apply_generic at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2940

@ROS34PW at /home/pvinod_umassd_edu/.julia/packages/OrdinaryDiffEq/EQy59/src/generic_rosenbrock.jl:970

jl_invoke_julia_macro at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/ast.c:1036

jl_expand_macros at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/ast.c:1092

ijl_expand_with_loc_warn at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/ast.c:1198

jl_toplevel_eval_flex at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/toplevel.c:709

jl_toplevel_eval_flex at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/toplevel.c:856

ijl_toplevel_eval_in at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/toplevel.c:971

eval at ./boot.jl:370 [inlined]

include_string at ./loading.jl:1903

_jl_invoke at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2758 [inlined]

ijl_apply_generic at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2940

_include at ./loading.jl:1963

include at ./Base.jl:457

jfptr_include_35036.clone_1 at /home/pvinod_umassd_edu/julia-1.9.3/lib/julia/sys.so (unknown line)

_jl_invoke at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2758 [inlined]

ijl_apply_generic at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2940

jl_apply at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/julia.h:1880 [inlined]

jl_f__call_latest at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/builtins.c:774

include at /home/pvinod_umassd_edu/.julia/packages/OrdinaryDiffEq/EQy59/src/OrdinaryDiffEq.jl:4

unknown function (ip: 0x7f31d637f7e2)

_jl_invoke at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2758 [inlined]

ijl_apply_generic at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2940

jl_apply at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/julia.h:1880 [inlined]

do_call at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/interpreter.c:126

eval_value at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/interpreter.c:226

eval_stmt_value at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/interpreter.c:177 [inlined]

eval_body at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/interpreter.c:624

jl_interpret_toplevel_thunk at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/interpreter.c:762

jl_toplevel_eval_flex at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/toplevel.c:912

jl_eval_module_expr at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/toplevel.c:203 [inlined]

jl_toplevel_eval_flex at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/toplevel.c:715

jl_toplevel_eval_flex at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/toplevel.c:856

jl_toplevel_eval_flex at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/toplevel.c:856

ijl_toplevel_eval_in at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/toplevel.c:971

eval at ./boot.jl:370 [inlined]

include_string at ./loading.jl:1903

_jl_invoke at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2758 [inlined]

ijl_apply_generic at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2940

_include at ./loading.jl:1963

include at ./Base.jl:457 [inlined]

include_package_for_output at ./loading.jl:2049

jfptr_include_package_for_output_38866.clone_1 at /home/pvinod_umassd_edu/julia-1.9.3/lib/julia/sys.so (unknown line)

_jl_invoke at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2758 [inlined]

ijl_apply_generic at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2940

jl_apply at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/julia.h:1880 [inlined]

do_call at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/interpreter.c:126

eval_value at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/interpreter.c:226

eval_stmt_value at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/interpreter.c:177 [inlined]

eval_body at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/interpreter.c:624

jl_interpret_toplevel_thunk at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/interpreter.c:762

jl_toplevel_eval_flex at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/toplevel.c:912

jl_toplevel_eval_flex at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/toplevel.c:856

ijl_toplevel_eval_in at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/toplevel.c:971

eval at ./boot.jl:370 [inlined]

include_string at ./loading.jl:1903

include_string at ./loading.jl:1913 [inlined]

exec_options at ./client.jl:305

_start at ./client.jl:522

jfptr__start_40034.clone_1 at /home/pvinod_umassd_edu/julia-1.9.3/lib/julia/sys.so (unknown line)

_jl_invoke at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2758 [inlined]

ijl_apply_generic at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2940

jl_apply at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/julia.h:1880 [inlined]

true_main at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/jlapi.c:573

jl_repl_entrypoint at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/jlapi.c:717

main at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/cli/loader_exe.c:59

__libc_start_main at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)

unknown function (ip: 0x4010b8)

Allocations: 13034676 (Pool: 13030710; Big: 3966); GC: 18

**ERROR:** LoadError: Failed to precompile OrdinaryDiffEq [1dea7af3-3e70-54e6-95c3-0bf5283fa5ed] to "/home/pvinod_umassd_edu/.julia/compiled/v1.9/OrdinaryDiffEq/jl_Uy2pmp".

Stacktrace:

[1] **error(**s::String**)**

@ Base ./error.jl:35

[2] **compilecache(**pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool**)**

@ Base ./loading.jl:2300

[3] **compilecache**

@ ./loading.jl:2167 [inlined]

[4] **_require(**pkg::Base.PkgId, env::String**)**

@ Base ./loading.jl:1805

[5] **_require_prelocked(**uuidkey::Base.PkgId, env::String**)**

@ Base ./loading.jl:1660

[6] **macro expansion**

@ ./loading.jl:1648 [inlined]

[7] **macro expansion**

@ ./lock.jl:267 [inlined]

[8] **require(**into::Module, mod::Symbol**)**

@ Base ./loading.jl:1611

[9] **include(**fname::String**)**

@ Base.MainInclude ./client.jl:478

[10] top-level scope

@ REPL[1]:1

in expression starting at /home/pvinod_umassd_edu/Direct_Comparision/GPU_runs/initializations.jl:18

Any help on what I’m missing is appreciated,
Pranav

What is the output of

using Pkg
Pkg.status()

?
And which CPUs are you using in your cluster? Are all the same?

I get this output for

Pkg.status()
**Status** `~/Direct_Comparision/GPU_runs/Project.toml`

[336ed68f] CSV v0.10.11

⌃ [2445eb08] DataDrivenDiffEq v1.1.0

⌃ [a93c6f00] DataFrames v1.5.0

⌃ [aae7a2af] DiffEqFlux v1.53.0

[41bf760c] DiffEqSensitivity v6.79.0

[d3d80556] LineSearches v7.2.0

⌅ [b2108857] Lux v0.4.51

⌃ [961ee093] ModelingToolkit v8.50.0

⌃ [429524aa] Optim v1.7.6

⌃ [1dea7af3] OrdinaryDiffEq v6.51.2

⌃ [91a5bcdd] Plots v1.38.16

⌃ [1ed8b502] SciMLSensitivity v7.29.0

⌃ [e88e6eb3] Zygote v0.6.65

[37e2e46d] LinearAlgebra

**Info** Packages marked with ⌃ and ⌅ have new versions available, but those with ⌅ are restricted by compatibility constraints from upgrading. To see why use `status --outdated`

I’m trying to run this on GPU, and I do believe I’m using the NVIDIA A100 and they are the same.

Remove DiffEqSensitivity. That was deprecated a long time ago. What led you to add it?

This suggests some heterogeneous CPUs in the cluster. You may need to use rhe environment variable JULIA_CPU_TARGET to force Julia to compile for multiple architectures when caching to disk.

Another workaround is to use --pkgimages=no command line option.

I haven’t updated my code in some time and this might have been there since the beginning.
It is a little weird, I shut my session on the cluster and restarted it and my program compiled successfully.
A few days later I encountered a similar error even on removing the DiffEqSensitivity package, that went away on restarting the session. Currently just working and hoping this doesn’t show up again.