Julia Crash Fatal Error

Specific Domains Optimization (Mathematical)
1

I keep encountering a fatal error in julia and can not solve it. The reproduction steps are too much to cover in here. The gist of my code is that I am trying to solve a LP optimization problem with slight changes over and over and record some results (A “while” loop within a “for” loop). However, no matter the julia version (1.3,1.5), or if I am running it with Atom or JuliaPro (1.4,1.5), the julia breaks with a fatal error:

Please submit a bug report with steps to reproduce this fault, and any error messages that follow (in their entirety). Thanks.
Exception: EXCEPTION_ACCESS_VIOLATION at 0x6685fb64 -- jl_is_concrete_immutable at /cygdrive/d/buildbot/worker/package_win64/build/src\cgutils.cpp:546 [inlined]
julia_type_to_llvm at /cygdrive/d/buildbot/worker/package_win64/build/src\cgutils.cpp:546
in expression starting at C:\Users\Reza\Documents\GitHub\create_data\uc_random.jl:42
jl_is_concrete_immutable at /cygdrive/d/buildbot/worker/package_win64/build/src\cgutils.cpp:546 [inlined]
julia_type_to_llvm at /cygdrive/d/buildbot/worker/package_win64/build/src\cgutils.cpp:546
mark_julia_const at /cygdrive/d/buildbot/worker/package_win64/build/src\codegen.cpp:657
emit_expr at /cygdrive/d/buildbot/worker/package_win64/build/src\codegen.cpp:4111
emit_invoke at /cygdrive/d/buildbot/worker/package_win64/build/src\codegen.cpp:3313
emit_expr at /cygdrive/d/buildbot/worker/package_win64/build/src\codegen.cpp:4139
emit_ssaval_assign at /cygdrive/d/buildbot/worker/package_win64/build/src\codegen.cpp:3851
emit_stmtpos at /cygdrive/d/buildbot/worker/package_win64/build/src\codegen.cpp:4044 [inlined]
emit_function at /cygdrive/d/buildbot/worker/package_win64/build/src\codegen.cpp:6671
jl_compile_linfo at /cygdrive/d/buildbot/worker/package_win64/build/src\codegen.cpp:1257
emit_invoke at /cygdrive/d/buildbot/worker/package_win64/build/src\codegen.cpp:3323
emit_expr at /cygdrive/d/buildbot/worker/package_win64/build/src\codegen.cpp:4139
emit_ssaval_assign at /cygdrive/d/buildbot/worker/package_win64/build/src\codegen.cpp:3851
emit_stmtpos at /cygdrive/d/buildbot/worker/package_win64/build/src\codegen.cpp:4044 [inlined]
emit_function at /cygdrive/d/buildbot/worker/package_win64/build/src\codegen.cpp:6671
jl_compile_linfo at /cygdrive/d/buildbot/worker/package_win64/build/src\codegen.cpp:1257
jl_compile_method_internal at /cygdrive/d/buildbot/worker/package_win64/build/src\gf.c:1889
_jl_invoke at /cygdrive/d/buildbot/worker/package_win64/build/src\gf.c:2153 [inlined]
jl_apply_generic at /cygdrive/d/buildbot/worker/package_win64/build/src\gf.c:2322
collect at .\array.jl:670
map at .\abstractarray.jl:2098
container at C:\Users\Reza\.juliapro\JuliaPro_v1.4.1-1\packages\JuMP\YXK4e\src\Containers\container.jl:85
container at C:\Users\Reza\.juliapro\JuliaPro_v1.4.1-1\packages\JuMP\YXK4e\src\Containers\container.jl:65
jl_apply at /cygdrive/d/buildbot/worker/package_win64/build/src\julia.h:1700 [inlined]
do_call at /cygdrive/d/buildbot/worker/package_win64/build/src\interpreter.c:369
eval_value at /cygdrive/d/buildbot/worker/package_win64/build/src\interpreter.c:458
eval_stmt_value at /cygdrive/d/buildbot/worker/package_win64/build/src\interpreter.c:409 [inlined]
eval_body at /cygdrive/d/buildbot/worker/package_win64/build/src\interpreter.c:799
jl_interpret_toplevel_thunk at /cygdrive/d/buildbot/worker/package_win64/build/src\interpreter.c:911
jl_toplevel_eval_flex at /cygdrive/d/buildbot/worker/package_win64/build/src\toplevel.c:814
jl_parse_eval_all at /cygdrive/d/buildbot/worker/package_win64/build/src\ast.c:872
jl_load at /cygdrive/d/buildbot/worker/package_win64/build/src\toplevel.c:872 [inlined]
jl_load_ at /cygdrive/d/buildbot/worker/package_win64/build/src\toplevel.c:879
include at .\client.jl:439
top-level scope at C:\Users\Reza\Documents\GitHub\MIP_Learning\create_data\extract_features.jl:16
jl_toplevel_eval_flex at /cygdrive/d/buildbot/worker/package_win64/build/src\toplevel.c:808
jl_parse_eval_all at /cygdrive/d/buildbot/worker/package_win64/build/src\ast.c:872
include_string at .\loading.jl:1080
include_string at C:\Users\Reza\.juliapro\JuliaPro_v1.4.1-1\packages\CodeTools\VsjEq\src\eval.jl:30
unknown function (ip: 000000001B189119)
#206 at C:\Users\Reza\.juliapro\JuliaPro_v1.4.1-1\packages\Atom\isnka\src\eval.jl:119
withpath at C:\Users\Reza\.juliapro\JuliaPro_v1.4.1-1\packages\CodeTools\VsjEq\src\utils.jl:30
withpath at C:\Users\Reza\.juliapro\JuliaPro_v1.4.1-1\packages\Atom\isnka\src\eval.jl:9
#205 at C:\Users\Reza\.juliapro\JuliaPro_v1.4.1-1\packages\Atom\isnka\src\eval.jl:118 [inlined]
with_logstate at .\logging.jl:398
with_logger at .\logging.jl:505 [inlined]
#204 at C:\Users\Reza\.juliapro\JuliaPro_v1.4.1-1\packages\Atom\isnka\src\eval.jl:117 [inlined]
hideprompt at C:\Users\Reza\.juliapro\JuliaPro_v1.4.1-1\packages\Atom\isnka\src\repl.jl:127
macro expansion at C:\Users\Reza\.juliapro\JuliaPro_v1.4.1-1\packages\Atom\isnka\src\eval.jl:116 [inlined]
macro expansion at C:\Users\Reza\.juliapro\JuliaPro_v1.4.1-1\packages\Media\ItEPc\src\dynamic.jl:24 [inlined]
eval at C:\Users\Reza\.juliapro\JuliaPro_v1.4.1-1\packages\Atom\isnka\src\eval.jl:113
unknown function (ip: 00000000022D6CF3)
jl_apply at /cygdrive/d/buildbot/worker/package_win64/build/src\julia.h:1700 [inlined]
do_apply at /cygdrive/d/buildbot/worker/package_win64/build/src\builtins.c:643
jl_f__apply at /cygdrive/d/buildbot/worker/package_win64/build/src\builtins.c:657 [inlined]
jl_f__apply_latest at /cygdrive/d/buildbot/worker/package_win64/build/src\builtins.c:693
#invokelatest#1 at .\essentials.jl:712
jl_apply at /cygdrive/d/buildbot/worker/package_win64/build/src\julia.h:1700 [inlined]
do_apply at /cygdrive/d/buildbot/worker/package_win64/build/src\builtins.c:643
invokelatest at .\essentials.jl:711
jl_apply at /cygdrive/d/buildbot/worker/package_win64/build/src\julia.h:1700 [inlined]
do_apply at /cygdrive/d/buildbot/worker/package_win64/build/src\builtins.c:643
macro expansion at C:\Users\Reza\.juliapro\JuliaPro_v1.4.1-1\packages\Atom\isnka\src\eval.jl:41 [inlined]
#188 at .\task.jl:358
unknown function (ip: 000000001B0CB9F3)
jl_apply at /cygdrive/d/buildbot/worker/package_win64/build/src\julia.h:1700 [inlined]
start_task at /cygdrive/d/buildbot/worker/package_win64/build/src\task.c:687
Allocations: 2462962494 (Pool: 2462406418; Big: 556076); GC: 458

I am printing the code and it seems to occur at around the same iteration counter each time.

 [fbb218c0] BSON v0.2.6       
  [052768ef] CUDA v1.2.1       
  [2b5f629d] DiffEqBase v6.41.3
  [aae7a2af] DiffEqFlux v1.18.0
  [0c46a032] DifferentialEquations v6.15.0
  [587475ba] Flux v0.11.0
  [38e38edf] GLM v1.3.9
  [7e08b658] GeometricFlux v0.6.1
  [2e9cd046] Gurobi v0.8.1
  [82e4d734] ImageIO v0.3.0
  [6218d12a] ImageMagick v1.1.5
  [916415d5] Images v0.22.4
  [c601a237] Interact v0.10.3
  [4138dd39] JLD v0.10.0
  [033835bb] JLD2 v0.1.14
  [4076af6c] JuMP v0.21.3
  [093fc24a] LightGraphs v1.3.3
  [b8f27783] MathOptInterface v0.9.14
  [fdba3010] MathProgBase v0.7.8
  [a4795742] NLPModels v0.12.4
  [792afdf1] NLPModelsJuMP v0.6.3
  [c36e90e8] PowerModels v0.17.2
  [438e738f] PyCall v1.91.4
  [dca85d43] QuartzImageIO v0.7.2
  [82193955] SCIP v0.9.6
  [47aef6b3] SimpleWeightedGraphs v1.1.1
  [789caeaf] StochasticDiffEq v6.25.0
  [9f7883ad] Tracker v0.2.8
  [98b73d46] Trebuchet v0.2.1
  [e88e6eb3] Zygote v0.5.4
  [37e2e46d] LinearAlgebra
  [10745b16] Statistics

Commit 381693d3df* (2020-04-14 17:20 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: Intel(R) Core(TM) i7-9700K CPU @ 3.60GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-8.0.1 (ORCJIT, skylake)
Environment:
  JULIA_DEPOT_PATH = C:\Users\Reza\.juliapro\JuliaPro_v1.4.1-1;C:\Users\Reza\AppData\Local\JuliaPro-1.4.1-1\Julia-1.4.1\local\share\julia;C:\Users\Reza\AppData\Local\JuliaPro-1.4.1-1\Julia-1.4.1\share\julia
  JULIA_EDITOR = "C:\Users\Reza\AppData\Local\JuliaPro-1.4.1-1\app-1.45.0\atom.exe"  -a
  JULIA_NUM_THREADS = 4
  JULIA_PKG_SERVER = pkg.juliacomputing.com
2

I doubt that we can solve it without any piece of code :confused:

Did you try to reduce your problem? Does it occur all the time?

3

It is very unfortunate, but I can not produce this fatal error with minimal code. For smaller size of iterations julia does not crash. I could receive different error messages while crashing though, with different solvers/versions.
one is this 1 line message:

julia LLVM ERROR: starting new .cfi frame before finishing the previous one

another one is the same as the one I posted except for their first few lines before “emit_expr”:

Assertion failed: jl_is_method_instance(mi), file /cygdrive/c/buildbot/worker/package_win64/build/src/codegen.cpp, line 2770

signal (22): SIGABRT
in expression starting at C:\Users\Reza\Documents\Julia_Dir\create_data\uc_random.jl:39
crt_sig_handler at /cygdrive/c/buildbot/worker/package_win64/build/src\signals-win.c:92
raise at C:\WINDOWS\System32\msvcrt.dll (unknown line)
abort at C:\WINDOWS\System32\msvcrt.dll (unknown line)
assert at C:\WINDOWS\System32\msvcrt.dll (unknown line)

That’s all I got on this issue.
I also tried running the same code on two other systems and it still occurs at the same iteration.

4

Moving this to Optimization (Mathematical) since it looks like it’s coming from JuMP’s container code.

Try isolating by using a Pkg environment (4. Working with Environments · Pkg.jl) containing only the packages you need (e.g., you probably don’t need ImageMagick if you’re solving optimization problems). (And upgrade to the latest versions. JuMP and Gurobi are both out-of-date.)

The most important thing is to reduce the number of packages involved., However, you can also try simplifying things by replacing data from external sources with hard-coded (or randomly generated) data, and commenting out as many variables/constraints as possible.

5

I have ran it with the latest packages on a freshly installed Julia 1.5 as well, which I believe contained the minimal set of required packages.

6

which I believe contained the minimal set of required packages.

What were they?

To be able to reproduce and then fix this, we need a minimal reproducible example that I can run. If you can share the example publicly, that’s great. If not, send me a private message.

7

These are status and version info. This crash is not solver dependent, it occurs with all (GLPK, Gurobi, CPLEX, Mosek).

C:\Users\Reza\.julia\environments\JuliaPro_v1.5.3-1\Project.toml
  [c52e3926] Atom v0.12.25 ⚲       
  [31c24e10] Distributions v0.24.12
  [60bf3e95] GLPK v0.14.6
  [7073ff75] IJulia v1.22.0        
  [4076af6c] JuMP v0.21.6
  [e5e0dc1b] Juno v0.8.4 ⚲
  [4722fa14] PkgAuthentication v0.3.0
  [c36e90e8] PowerModels v0.17.4
  [44d3d7a6] Weave v0.10.6
  [37e2e46d] LinearAlgebra
  [9a3f8284] Random

Commit 788b2c77c1 (2020-11-09 13:37 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: Intel(R) Core(TM) i5-3230M CPU @ 2.60GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-9.0.1 (ORCJIT, ivybridge)
Environment:
  JULIA_DEPOT_PATH = C:\Users\Reza\.julia;C:\Users\Reza\AppData\Local\JuliaPro-1.5.3-1\Julia-1.5.3\local\share\julia;C:\Users\Reza\AppData\Local\JuliaPro-1.5.3-1\Julia-1.5.3\share\julia
  JULIA_EDITOR = "C:\Users\Reza\AppData\Local\JuliaPro-1.5.3-1\app-1.53.0\atom.exe"  -a
  JULIA_LOAD_PATH = C:\Users\Reza\.julia\environments\JuliaPro_v1.5.3-1;@;@v#.#;@stdlib
  JULIA_NUM_THREADS = 2
  JULIA_PKG_SERVER = pkg.juliahub.com

I will work on minimal code and share with you if possible.

1 Like