# PyJulia multithreading segfaults

**URL:** <https://discourse.julialang.org/t/pyjulia-multithreading-segfaults/65111>\
**Category:** General Usage\
**Tags:** question, multithreading, juliacall, pythoncall\
**Created:** [July 22, 2021, 2:25pm UTC](https://discourse.julialang.org/t/pyjulia-multithreading-segfaults/65111 "2021-07-22T14:25:49Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![andrewkhardy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/andrewkhardy/32/27311_2.png) [@andrewkhardy](https://discourse.julialang.org/u/andrewkhardy)\
**Post date:** [July 22, 2021, 2:25pm UTC](https://discourse.julialang.org/t/pyjulia-multithreading-segfaults/65111/1 "2021-07-22T14:25:50Z")

</div>

I am wondering if the Julia community has any suggestions for importing code from Julia to Python in some manner other than the [PyJulia package](https://github.com/JuliaPy/pyjulia) or suggestions on how to properly use that package as the GitHub seems to be inactive.

I imagine a lot of the advice will be to rewrite the code in Julia, and while I’d like to do that eventually, I’d prefer to have a working solution soon and know this is not a Julia bug.

I am suffering from a segmentation fault 11 error when multithreading using [joblib.Parallel](https://joblib.readthedocs.io/en/latest/generated/joblib.Parallel.html#joblib.Parallel) to run a small Julia function.

I import a function in Python, modeled after [this attempted solution](https://github.com/JuliaPy/pyjulia/issues/306)

```julia
from julia.api import Julia
jl = Julia(compiled_modules=False)
from julia import Main
Main.include("fastsum.jl")
from julia.Main import greenfunction

```

where the function itself is

```julia
using Tullio, LoopVectorization
function greenfunction(mu, wns, sigwns, energy, dosnorm)
    new_array = Array{ComplexF64}(undef, length(wns))
    @tullio threads=false new_array[i] = 1 / (mu + wns[i] * 1im - energy[j] - sigwns[i]) * dosnorm[j]
    dosnorm = Nothing
    energy = Nothing
    sigwns = Nothing
    wns = Nothing
    
    return new_array
end

```

Clearing some of the variables seems to make the number of processes I can run increase from something around 10 to 80, but eventually it still segfaults, with error as

```julia
signal (15): Terminated
in expression starting at none:0
mul_fast at ./fastmath.jl:167 [inlined]
mul_fast at ./fastmath.jl:219 [inlined]
...
... ( a lot of PyCall directories)
...
unknown function (ip: (nil))
Allocations: 76861098 (Pool: 76845028; Big: 16070); GC: 116

signal (11): Segmentation fault
in expression starting at none:0
...
joblib.externals.loky.process_executor.TerminatedWorkerError: A worker process managed by the executor was unexpectedly terminated. This could be caused by a segmentation fault while calling the function or by an excessive memory usage causing the Operating System to kill the worker.

```

The more detailed errors are [here](https://github.com/JuliaPy/pyjulia/issues/362)

Is there a way to clear the Julia cache as this seems to be the problem? Or import Julia less frequently somehow? I call that import function several hundred times.

---

<div class="post-metadata">

**Author:** ![cjdoris](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cjdoris/32/213133_2.png) [@cjdoris](https://discourse.julialang.org/u/cjdoris)\
**Post date:** [July 23, 2021, 8:28am UTC](https://discourse.julialang.org/t/pyjulia-multithreading-segfaults/65111/2 "2021-07-23T08:28:30Z")

</div>

I would bet it’s a GIL problem. I don’t think PyCall / PyJulia handles the GIL at all.

Have you tried [PythonCall](https://github.com/cjdoris/PythonCall.jl) / juliacall?

It would also be helpful to include a full MWE.