Cannot run parallel for loop when calling R code

General Usage
,
1

Hi, I am recently moving my projects from R to Julia, and part of the code still relies on some R packages. I encountered problems when putting the R code inside the parallel for loop by using Threads.@threads. Below is a minimal working example:

using RCall

Threads.@threads for i in 1:10
    R"a = rnorm(1)"
    @rget a;
    println(a)
end

Julia terminated automatically and I got the following error message:

*** longjmp causes uninitialized stack frame ***: /path/.local/bin/julia terminated

signal (11): Segmentation fault
in expression starting at untitled-f2168f6cd01b7cefdedd9bf234089f00:3
*** longjmp causes uninitialized stack frame ***: /path/.local/bin/julia terminated

My operating system is Ubuntu 18.04 running Julia 1.5.0

I was wondering if there is anything I can do to call R functions inside a for loop in Julia? Thanks!

2

I’m not sure, but I bet it’s R that’s not thread safe here, so you’re not going to be able to do this.

You could use multiprocessing (running in separate processes).

3

Yes using multi-processing works! Thanks a lot!

4

It works when running on my computer, but I got the following errors when using the @distributed to initialize the for loop on a Linux server

Stacktrace:
 [1] sync_end(::Array{Any,1}) at ./task.jl:235
 [2] macro expansion at ./task.jl:254 [inlined]
 [3] #addprocs_locked#44(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{,Tuple{}}}, ::typeof(Distributed.addprocs_locked), ::Distributed.LocalManager) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.2/Distributed/src/cluster.jl:474
 [4] addprocs_locked at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.2/Distributed/src/cluster.jl:445 [inlined]
 [5] #addprocs#43(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{,Tuple{}}}, ::typeof(addprocs), ::Distributed.LocalManager) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.2/Distributed/src/cluster.jl:438
 [6] addprocs at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.2/Distributed/src/cluster.jl:432 [inlined]
 [7] #addprocs#253 at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.2/Distributed/src/managers.jl:316 [inlined]
 [8] addprocs(::Int64) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.2/Distributed/src/managers.jl:315
 [9] top-level scope at /u/y227xie/ResearchDocuments/fpp_julia/fcp_mean_fast.jl:11
 [10] include at ./boot.jl:328 [inlined]
 [11] include_relative(::Module, ::String) at ./loading.jl:1094
 [12] include(::Module, ::String) at ./Base.jl:31
 [13] exec_options(::Base.JLOptions) at ./client.jl:295
 [14] _start() at ./client.jl:464

Do you have any idea what was happening? Thanks!

5

Can you give the code you use? Hard to know what’s wrong without knowing the code.