I’m working on porting an in-house library from R to Julia. On the plus side I’m seeing greatly improved performance - in the best case a 1,000 times speed up for code that performs an optimisation using LNopt versus R code that uses dfsane.
But, not so “plus side”, I’m seeing quite frequent crashes of Julia, which request me to “Please submit a bug report”. The relevant documentation suggests that my first step should be to “Create a debug build of Julia, and run your script inside of a debugged Julia process” before uploading a backtrace to a gist and submitting a bug report.
I think that figuring out how to “create a debug build of Julia” would take me quite some time (my coding experience is all R, Matlab and VBA, so no compiled languages), but I do have the following output from the REPL. Would it be sufficient to “help the maintainers when tracking down a segfault”?
Please submit a bug report with steps to reproduce this fault, and any error messages that follow (in their entirety). Thanks.
Exception: EXCEPTION_ACCESS_VIOLATION at 0x0 -- unknown function (ip: 0000000000000000)
in expression starting at REPL[2]:1
unknown function (ip: FFFFFFFFFFFFFFFF)
unknown function (ip: 0000000001E308BA)
jl_apply_generic at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\gf.c:2191
findfirst at .\array.jl:1742
unknown function (ip: 0000000004EB8433)
jl_apply_generic at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\gf.c:2191
extractfxstate at d:\Philip\Source\script\Julia\JSource\xccyhwHooks.jl:86
BtStDt_xccyhw at d:\Philip\Source\script\Julia\JSource\xccyhwHooks.jl:295
fixedleg at d:\Philip\Source\script\Julia\JSource\Instruments.jl:160
unknown function (ip: 0000000001E28205)
jl_apply_generic at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\gf.c:2197
fixedleg at d:\Philip\Source\script\Julia\JSource\Instruments.jl:123
interestrateswap at d:\Philip\Source\script\Julia\JSource\Instruments.jl:216
unknown function (ip: 0000000001E2697D)
jl_apply_generic at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\gf.c:2197
valuetrade at d:\Philip\Source\script\Julia\JSource\UtilsValue.jl:46
macro expansion at d:\Philip\Source\script\Julia\JSource\UtilsValue.jl:17 [inlined]
#520#threadsfor_fun at .\threadingconstructs.jl:64
#520#threadsfor_fun at .\threadingconstructs.jl:31
jl_apply_generic at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\gf.c:2191
jl_apply at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\julia.h:1614 [inlined]
start_task at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\task.c:596
Allocations: 289202867 (Pool: 289124945; Big: 77922); GC: 675
Version info:
julia> versioninfo()
Julia Version 1.2.0
Commit c6da87ff4b (2019-08-20 00:03 UTC)
Platform Info:
OS: Windows (x86_64-w64-mingw32)
CPU: Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-6.0.1 (ORCJIT, skylake)
Environment:
JULIA_NUM_THREADS = 8
JULIA_EDITOR = "C:\Users\Philip\AppData\Local\Programs\Microsoft VS Code\Code.exe"
Please do file an issue at https://github.com/JuliaLang/Julia/issues, ideally with a description of how to reproduce. Core devs are very responsive to crash reports. It’s possible that the crash is due to some of the packages interacting with C, but we’d rather get that kind of big report and have to direct you elsewhere after figuring that out than risk not getting reports of crashes that are Julia’s fault.
Trying to reduce the problem as much as possible is helpful. For example, does it still happen when you don’t run things threaded? Can you reduce the function body while still getting the problem etc etc.
Thanks for the prompt reply. It’s good to know that effort I put in to reduce the problem and make the crash easy-to-replicate won’t be wasted, and Kristoffer asked a good question about threaded or not. I _think_the answer is that the crash does not happen when not threaded, but I need to make sure.
As things stand (no “problem reduction”) putting the core devs into a position to replicate the problem would involve sending approx 8,000 lines of code in 20 files, together with input data, and instructions as to how to run the code to make the crash happen.
One additional question: There is some commercial value to the code. Is there any way to send code privately so that it’s not available to anyone who logs on to the Julia Issues page?
I think the best course of action would be to kill two birds with one stone and try to reduce the problem to something much more minimal, say ~10-20 lines of code. That way you don’t have to worry about giving away your companies code and the devs don’t have to go through your whole code base and have a much easier time trying to find a solution.
Yes, I couldn’t agree more. My difficulty is that any “divide and conquer” approach to problem reduction is hampered by the fact that the crashes I’m seeing are quite intermittent. I get a crash, restart Julia, run (AFAICT) the same code and the crash does not happen. That means that it might be some time before I’m able to submit anything useful.
Yes, it’s definitely possible to just send the code directly to a couple of Julia devs who will look at it privately. I would file and issue first, including this information.
I experience very similar crashes, also under Windows. In my case they occur in connection with gmp: gmp_default_alloc_overflow. The unpleasant thing is that they are not reproducible once the code runs on Travis. (The same code seems to run without errors on Linux and OSX.)
Julia Version 1.3.0-rc4.1 Commit 8c4656b97a (2019-10-15 14:08 UTC)
Platform Info: OS: Windows (x86_64-w64-mingw32)
CPU: AMD Athlon(tm) II X4 640 Processor
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-6.0.1 (ORCJIT, amdfam10)
Environment: JULIA_EDITOR = "atom.exe" -a
JULIA_NUM_THREADS = 4
signal (22): SIGABRT
crt_sig_handler at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\signals-win.c:91
raise at C:\WINDOWS\System32\msvcrt.dll (unknown line)
abort at C:\WINDOWS\System32\msvcrt.dll (unknown line)
__gmp_default_alloc_overflow at C:\Users\Home\AppData\Local\Julia-1.3.0-rc4\bin\libgmp-10.dll (unknown line)
.text at C:\Users\Home\AppData\Local\Julia-1.3.0-rc4\bin\libgmp-10.dll (unknown line)
.text at C:\Users\Home\AppData\Local\Julia-1.3.0-rc4\bin\libgmp-10.dll (unknown line)
macro expansion at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.3\Test\src\Test.jl:1107 [inlined]
unknown function (ip: 0000000017BB4B03)
_jl_invoke at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\gf.c:2136 [inlined]
jl_apply_generic at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\gf.c:2300
jl_apply at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\julia.h:1631 [inlined]
do_call at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\interpreter.c:328
eval_value at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\interpreter.c:417
eval_stmt_value at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\interpreter.c:368 [inlined]
eval_body at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\interpreter.c:764
jl_interpret_toplevel_thunk_callback at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\interpreter.c:888
unknown function (ip: FFFFFFFFFFFFFFFE)
unknown function (ip: 0000000012CCDE8F)
unknown function (ip: FFFFFFFFFFFFFFFF)
jl_toplevel_eval_flex at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\toplevel.c:814
jl_eval_module_expr at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\toplevel.c:181
jl_toplevel_eval_flex at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\toplevel.c:640
jl_parse_eval_all at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\ast.c:873
include_string at .\loading.jl:1075 #139 at C:\Users\Home.julia\packages\Atom\lBERI\src\eval.jl:138
withpath at C:\Users\Home.julia\packages\CodeTools\xGemk\src\utils.jl:30
_jl_invoke at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\gf.c:2136 [inlined]
jl_apply_generic at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\gf.c:2300
withpath at C:\Users\Home.julia\packages\Atom\lBERI\src\eval.jl:47 #138 at C:\Users\Home.julia\packages\Atom\lBERI\src\eval.jl:135 [inlined]
with_logstate at .\logging.jl:395
with_logger at .\logging.jl:491 [inlined] #137 at C:\Users\Home.julia\packages\Atom\lBERI\src\eval.jl:134 [inlined]
hideprompt at C:\Users\Home.julia\packages\Atom\lBERI\src\repl.jl:85
_jl_invoke at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\gf.c:2136 [inlined]
jl_apply_generic at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\gf.c:2300
macro expansion at C:\Users\Home.julia\packages\Atom\lBERI\src\eval.jl:133 [inlined]
macro expansion at C:\Users\Home.julia\packages\Media\ItEPc\src\dynamic.jl:24 [inlined] #136 at C:\Users\Home.julia\packages\Atom\lBERI\src\eval.jl:122
_jl_invoke at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\gf.c:2136 [inlined]
jl_apply_generic at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\gf.c:2300
handlemsg at C:\Users\Home.julia\packages\Atom\lBERI\src\comm.jl:164
unknown function (ip: 0000000017B73432)
_jl_invoke at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\gf.c:2130 [inlined]
jl_apply_generic at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\gf.c:2300
jl_apply at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\julia.h:1631 [inlined]
jl_f__apply at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\builtins.c:627 #19 at .\task.jl:333
unknown function (ip: 0000000017B6FCAE)
_jl_invoke at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\gf.c:2130 [inlined]
jl_apply_generic at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\gf.c:2300
jl_apply at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\julia.h:1631 [inlined]
start_task at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\task.c:659
Allocations: 32613597 (Pool: 32606222; Big: 7375); GC: 36