Possible to define a module where one dependency is built after running other stuff first?

New to Julia
, ,
1

I need this module to create a Conda environment whose full path is required by Pkg.build() for another dependency.

tmpdir = "/home/me/Desktop/tmp"
mkdir(tmpdir)
cd(tmpdir)
using Pkg
Pkg.activate("./")
Pkg.add(["RCall","Conda","Dates"])

module BuildDepsSeparately

using Pkg
using Conda,Dates
using RCall

# install r to a temp path and return the path
function initR()
    # make a new temporary conda environment for r
    ts = Dates.time()
    tdir = joinpath(tmpdir,"r$ts")
    mkpath(tdir)
    run(`conda create -p $tdir -y`)

    # install r in the temp environment
    Conda.add_channel("r",tdir)
    Conda.add("r-base>=3.4.0,<4",tdir) # greater than or equal to 3.4.0 AND strictly less than 4.0

    ENV["R_HOME"] = joinpath(tdir,"lib","R")
    Pkg.build("RCall")
    return(tdir)
end
end

Running this code causes the REPL to crash with the following errors:

full error output 
[ Info: Precompiling RCall [6f49c342-dc21-5d91-9882-a32aef131414]
FATAL ERROR: Symbol "ccalllib_libR.so4725"not found
signal (6): Aborted
in expression starting at /home/me/Desktop/scratch3.jl:12
gsignal at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
abort at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
addModule at /buildworker/worker/package_linux64/build/src/jitlayers.cpp:640
jl_add_to_ee at /buildworker/worker/package_linux64/build/src/jitlayers.cpp:893 [inlined]
jl_add_to_ee at /buildworker/worker/package_linux64/build/src/jitlayers.cpp:955
jl_add_to_ee at /buildworker/worker/package_linux64/build/src/jitlayers.cpp:977 [inlined]
_jl_compile_codeinst at /buildworker/worker/package_linux64/build/src/jitlayers.cpp:126
jl_generate_fptr at /buildworker/worker/package_linux64/build/src/jitlayers.cpp:302
jl_compile_method_internal at /buildworker/worker/package_linux64/build/src/gf.c:1964
gen_cfun_wrapper at /buildworker/worker/package_linux64/build/src/codegen.cpp:4068
emit_cfunction at /buildworker/worker/package_linux64/build/src/codegen.cpp:4654 [inlined]
emit_expr at /buildworker/worker/package_linux64/build/src/codegen.cpp:3694
emit_ssaval_assign at /buildworker/worker/package_linux64/build/src/codegen.cpp:3339
emit_stmtpos at /buildworker/worker/package_linux64/build/src/codegen.cpp:3576 [inlined]
emit_function at /buildworker/worker/package_linux64/build/src/codegen.cpp:6110
jl_emit_code at /buildworker/worker/package_linux64/build/src/codegen.cpp:6463
jl_emit_codeinst at /buildworker/worker/package_linux64/build/src/codegen.cpp:6497
_jl_compile_codeinst at /buildworker/worker/package_linux64/build/src/jitlayers.cpp:97
jl_generate_fptr at /buildworker/worker/package_linux64/build/src/jitlayers.cpp:302
jl_compile_method_internal at /buildworker/worker/package_linux64/build/src/gf.c:1964
jl_compile_method_internal at /buildworker/worker/package_linux64/build/src/gf.c:1919 [inlined]
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2224 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2398
jl_apply at /buildworker/worker/package_linux64/build/src/julia.h:1690 [inlined]
jl_module_run_initializer at /buildworker/worker/package_linux64/build/src/toplevel.c:74
jl_init_restored_modules at /buildworker/worker/package_linux64/build/src/dump.c:2544
_include_from_serialized at ./loading.jl:697
_require_from_serialized at ./loading.jl:749
_require at ./loading.jl:1040
require at ./loading.jl:928
require at ./loading.jl:923
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2214 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2398
jl_apply at /buildworker/worker/package_linux64/build/src/julia.h:1690 [inlined]
call_require at /buildworker/worker/package_linux64/build/src/toplevel.c:425 [inlined]
eval_import_path at /buildworker/worker/package_linux64/build/src/toplevel.c:462
jl_toplevel_eval_flex at /buildworker/worker/package_linux64/build/src/toplevel.c:682
jl_eval_module_expr at /buildworker/worker/package_linux64/build/src/toplevel.c:197
jl_toplevel_eval_flex at /buildworker/worker/package_linux64/build/src/toplevel.c:666
jl_parse_eval_all at /buildworker/worker/package_linux64/build/src/ast.c:913
include_string at ./loading.jl:1091
include_string at /home/me/.julia/packages/Atom/9OFfu/src/utils.jl:280
unknown function (ip: 0x7f3ca67f5b26)
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2214 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2398
#190 at /home/me/.julia/packages/Atom/9OFfu/src/eval.jl:76
withpath at /home/me/.julia/packages/CodeTools/VsjEq/src/utils.jl:30
unknown function (ip: 0x7f3ca67f59f5)
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2214 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2398
withpath at /home/me/.julia/packages/Atom/9OFfu/src/eval.jl:9
#189 at /home/me/.julia/packages/Atom/9OFfu/src/eval.jl:74
unknown function (ip: 0x7f3ca67f51ac)
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2214 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2398
with_logstate at ./logging.jl:408
with_logger at ./logging.jl:514 [inlined]
#188 at /home/me/.julia/packages/Atom/9OFfu/src/eval.jl:73 [inlined]
hideprompt at /home/me/.julia/packages/Atom/9OFfu/src/repl.jl:127
macro expansion at /home/me/.julia/packages/Atom/9OFfu/src/eval.jl:72 [inlined]
macro expansion at /home/me/.julia/packages/Media/ItEPc/src/dynamic.jl:24 [inlined]
evalshow at /home/me/.julia/packages/Atom/9OFfu/src/eval.jl:69
unknown function (ip: 0x7f3ca67e55e2)
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2214 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2398
jl_apply at /buildworker/worker/package_linux64/build/src/julia.h:1690 [inlined]
do_apply at /buildworker/worker/package_linux64/build/src/builtins.c:655
jl_f__apply_latest at /buildworker/worker/package_linux64/build/src/builtins.c:705
#invokelatest#1 at ./essentials.jl:710
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2214 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2398
jl_apply at /buildworker/worker/package_linux64/build/src/julia.h:1690 [inlined]
do_apply at /buildworker/worker/package_linux64/build/src/builtins.c:655
invokelatest at ./essentials.jl:709
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2214 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2398
jl_apply at /buildworker/worker/package_linux64/build/src/julia.h:1690 [inlined]
do_apply at /buildworker/worker/package_linux64/build/src/builtins.c:655
macro expansion at /home/me/.julia/packages/Atom/9OFfu/src/eval.jl:41 [inlined]
#184 at ./task.jl:356
unknown function (ip: 0x7f3ca670458c)
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2231 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2398
jl_apply at /buildworker/worker/package_linux64/build/src/julia.h:1690 [inlined]
start_task at /buildworker/worker/package_linux64/build/src/task.c:707
unknown function (ip: (nil))
Allocations: 50354887 (Pool: 50338039; Big: 16848); GC: 53

Julia has exited.

This appears to be a result of some issue with the module declaration because the following test passes:

tmpdir = "/home/me/Desktop/tmp"
mkpath(tmpdir)
cd(tmpdir)
using Pkg
Pkg.activate("./")
Pkg.add(["RCall","Conda","Dates"])
using Conda,Dates

ts = Dates.time()
tdir = joinpath(tmpdir,"r$ts")
mkpath(tdir)
run(`conda create -p $tdir -y`)

# install r in the temp environment
Conda.add_channel("r",tdir)
Conda.add("r-base>=3.4.0,<4",tdir) # greater than or equal to 3.4.0 AND strictly less than 4.0

ENV["R_HOME"] = joinpath(tdir,"lib","R")
Pkg.build("RCall")

using RCall

R"wd = getwd()"
dat = rcopy(R"wd")
run(`conda env remove -p $tdir`)


using Test
@test dat == tmpdir

Test Passed

–edit updated for clarity
–edit fix example bug (not related to error)
–edit new examples and update crash output

2

What exactly you want to know if it is possible? I understand you already have a package that does that, so what exactly is the doubt?

3

@Henrique_Becker sorry the post was unclear, I have updated to explain what I need

4

The error seems related to building RCall, no? Did you try write some code that just try to build RCall with a ENV["R_HOME"] you created manually and are sure it is alright? This ccalllib_libR.so4630 is somewhere in the folder Conda created?

5

Yes it works if I force RCall to create its own Conda environment by defining:

ENV[R_HOME] = "*"
Pkg.build("RCall")
6

Can you run diff -R <your_folder> <auto_rcall_folder> to check the differences between the environment you created by hand and the one RCall creates automatically? Also, maybe you can check how Rcall does so (by checking its source).

1 Like
7

Hi @Henrique_Becker, I’ve updated the example, the error, and added an MWE unit test for the code the module attempts to run to clarify what is happening.