I train some rnn model and also use pyjulia to get some word embeddings during training, i.e. for each batch I use pyjulia to get word embeddings. First 5-6 epochs all is ok, but then I get the segfault error, it is strange because each epoch julia does the same things for same batches.
Linux Mint 17.3 Rosa
Linux server 3.19.0-32-generic #37~14.04.1-Ubuntu SMP Thu Oct 22 09:41:40 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
pytorch ‘0.3.0.post4’
pyjulia 0.1.5
Julia v0.5
Julia PyCall 1.15.0
Error is:
signal (11): Segmentation fault
while loading no file, in expression starting on line 0
GetResult at /home/ilan/minonda/conda-bld/python_1494526091235/work/Python-3.6.1/Modules/_ctypes/callproc.c:909 [inlined]
_ctypes_callproc at /home/ilan/minonda/conda-bld/python_1494526091235/work/Python-3.6.1/Modules/_ctypes/callproc.c:1184
PyCFuncPtr_call at /home/ilan/minonda/conda-bld/python_1494526091235/work/Python-3.6.1/Modules/_ctypes/_ctypes.c:3870
_PyObject_FastCallDict at /home/ilan/minonda/conda-bld/python_1494526091235/work/Python-3.6.1/Objects/abstract.c:2316
call_function at /home/ilan/minonda/conda-bld/python_1494526091235/work/Python-3.6.1/Python/ceval.c:4822
_PyEval_EvalFrameDefault at /home/ilan/minonda/conda-bld/python_1494526091235/work/Python-3.6.1/Python/ceval.c:3284
_PyFunction_FastCall at /home/ilan/minonda/conda-bld/python_1494526091235/work/Python-3.6.1/Python/ceval.c:4880
fast_function at /home/ilan/minonda/conda-bld/python_1494526091235/work/Python-3.6.1/Python/ceval.c:4915 [inlined]
call_function at /home/ilan/minonda/conda-bld/python_1494526091235/work/Python-3.6.1/Python/ceval.c:4819
_PyEval_EvalFrameDefault at /home/ilan/minonda/conda-bld/python_1494526091235/work/Python-3.6.1/Python/ceval.c:3284
_PyFunction_FastCall at /home/ilan/minonda/conda-bld/python_1494526091235/work/Python-3.6.1/Python/ceval.c:4880
fast_function at /home/ilan/minonda/conda-bld/python_1494526091235/work/Python-3.6.1/Python/ceval.c:4915 [inlined]
call_function at /home/ilan/minonda/conda-bld/python_1494526091235/work/Python-3.6.1/Python/ceval.c:4819
_PyEval_EvalFrameDefault at /home/ilan/minonda/conda-bld/python_1494526091235/work/Python-3.6.1/Python/ceval.c:3284
_PyEval_EvalCodeWithName at /home/ilan/minonda/conda-bld/python_1494526091235/work/Python-3.6.1/Python/ceval.c:4128
fast_function at /home/ilan/minonda/conda-bld/python_1494526091235/work/Python-3.6.1/Python/ceval.c:4939 [inlined]
call_function at /home/ilan/minonda/conda-bld/python_1494526091235/work/Python-3.6.1/Python/ceval.c:4819
_PyEval_EvalFrameDefault at /home/ilan/minonda/conda-bld/python_1494526091235/work/Python-3.6.1/Python/ceval.c:3300
gen_send_ex at /home/ilan/minonda/conda-bld/python_1494526091235/work/Python-3.6.1/Objects/genobject.c:189
_PyEval_EvalFrameDefault at /home/ilan/minonda/conda-bld/python_1494526091235/work/Python-3.6.1/Python/ceval.c:3060
_PyEval_EvalCodeWithName at /home/ilan/minonda/conda-bld/python_1494526091235/work/Python-3.6.1/Python/ceval.c:4128
PyEval_EvalCodeEx at /home/ilan/minonda/conda-bld/python_1494526091235/work/Python-3.6.1/Python/ceval.c:4149
PyEval_EvalCode at /home/ilan/minonda/conda-bld/python_1494526091235/work/Python-3.6.1/Python/ceval.c:695
run_mod at /home/ilan/minonda/conda-bld/python_1494526091235/work/Python-3.6.1/Python/pythonrun.c:980 [inlined]
PyRun_FileExFlags at /home/ilan/minonda/conda-bld/python_1494526091235/work/Python-3.6.1/Python/pythonrun.c:933
PyRun_SimpleFileExFlags at /home/ilan/minonda/conda-bld/python_1494526091235/work/Python-3.6.1/Python/pythonrun.c:396
run_file at /home/ilan/minonda/conda-bld/python_1494526091235/work/Python-3.6.1/Modules/main.c:338 [inlined]
Py_Main at /home/ilan/minonda/conda-bld/python_1494526091235/work/Python-3.6.1/Modules/main.c:810
main at python (unknown line)
__libc_start_main at /build/eglibc-SvCtMH/eglibc-2.19/csu/libc-start.c:287
unknown function (ip: 0x4009e8)
Allocations: 896429807 (Pool: 896428379; Big: 1428); GC: 932