I’m trying to create an app using PackageCompiler.jl, but my app depends on an R package, where I pass variables between Julia and R using RCall.jl.
To create this app, I tried a minimal example that just puts some strings into R via RCall.jl and prints them.
module Myapp
using RCall
function julia_main()::Cint
arg1 = ARGS[1]
@rput arg1
R"""
print("Myapp: reached within R from Julia")
print(arg1)
"""
return 0
end
end # module Myapp
Creating the app with create_app(Myapp, Myapp_compiled) worked fine, but the app doesn’t work on another machine:
$ ./Myapp 123
ERROR: could not load library "/share/software/user/open/R/4.0.2/lib64/R/lib/libR.so"
libRblas.so: cannot open shared object file: No such file or directory
fatal: error thrown and no exception handler available.
I think it means PackageCompiler.jl is not bundling R with the app, and is trying to invoke the R from my local installation. Is there any workarounds to this issue?
This is the same as this issue but it doesn’t seem to be getting any traffic, so I’m hoping the community can give me some opinions.
I think you’re expecting too much. R is GPL-licenced, and while there’s hypothetically nothing wrong with distributing/conveying it with, and/or R package dependencies, and many or most are GPL licenced, it could be in violation if some of your Julia package dependencies are non-GPL compatible, and you distribute ALL that’s needed.
If you do not distribute R with then, it may not be useless, since RCall, that you would distribute with looks for it, and if it finds it then you’re good. That’s assuming your R packages are also preinstalled there already.
You could hypothetically distribute the R packages in you compiled Julia program (artefacts? but I’m pretty sure it’s not something PackageCompiler.jl does for you). Or download them on first use. Downloading on first use would be a legal grey area if everything you package isn’t GPL-compatible.
If R (and R packages likely) ARE preinstalled at your target then it might fall under the system library exception (even if you were to use proprietary code/non-GPL).
If you distribute your code with (GPL components, e.g. R itself), then your own code must be free/open source software, GPL-compatible, if not using GPL itself.
What you do in private with any GPL code is up to you, you’re are NOT breaking the GPL if you use it in any way or send people a binary (or source code) that links to it (without download GPL code at least for you [automatically]).
I don’t know an easy way to use PackageCompiler.jl either, but despite R and Python not using Julia’s package manager, the difference with R is that PythonCall.jl will download Python for you if needed, but it also takes care of Python dependencies (by downloading them, so you could do that, just have a file listing them, or figuring out to include all of them in the compiled package). Unlike R, GPL Python packages aren’t as common.
So while this is technically possible, also for R, I just don’t know if anyone has done it yet. If you find a replacement Julia package for your R package, that might be best. Next best likely a replacement Python package. I’m curious, which R package is this, and is everything else GPL-compatible (e.g. MIT licenced)?
This R package is a research software, which wraps a bunch of C++ code. It is not publically available yet. If I want to drop R (sounds like that’s what I need to do), I can potentially write a Julia wrapper for those C++ code. That way, I can put it into a jll package, host it on Yggdrasil, and within PackageCompiler.jl treat it as an Artifact. Then PackageCompiler.jl will hopefully work, but fingers crossed as I’ve never done this before…
As I said it IS legal to distribute GPL code (just practically technically difficult to package R [code] and Julia together, and not because it’s GPL).
But I want to add one thing on that. If you do compile (to hide code, one reason for some; for speed is another but it doesn’t really apply to Julia, since Julia is compiled already, and packages precompiled fully since 1.9), then the application “as a whole” is under GPL, and you must distribute source code with the app.
In GPLv2 that were the rules, OR provide a written offer to provide the source. For GPLv3 things changed a bit quite a bit. People often disregard those rules, do not bundle the source code. It might be enough to provide a link to the source code (e.g. on Github?).
If you’re linking to GPL code you got from someone (e.g. open source), it’s still your resonsibility to make sure that link stays up forever, so in practice, if you want to be strict, it means you copy the code to your web server. At that point it might be better to just bundle the source code.
But do you really want to bundle the full source code for even R itself…?
A written offer might be better since it expires in three years. That’s for GPL v2. I think that option is still there in GPL v3, but now it’s limited to noncommercial use, or otherwise a lot more onerous.
There are some R packages with a Julia wrapper, at least this one (and see my Doc/license PR):
There might not be many such R wrapper packages yet (also strictly not needed, as you can always do without use them directly, with RCall).
There are also some Julia packages wrapping Python code (probably a lot more of those, though still limited amount of those using the newer PythonCall.jl, I list these two):
I don’t know if Conda is used a lot in the R world. It’s not the default package manager (neither for Python really, though used a lot), and you can see the Conda files for the dependencies on the relevant packages. Possibly RCall should do the same as PythonCall, use Conda. That simplifies installation, which maybe was your main goal, just to make installation convenient for your users?
For GPLv2 you have the written offer option, limited for three years, but R changed to GPL v3, and then it’s forever basically, or for non-commercial:
6. Conveying Non-Source Forms.
You may convey a covered work in object code form under the terms of sections 4 and 5, provided that you also convey the machine-readable Corresponding Source under the terms of this License, in one of these ways:
a) Convey the object code in, or embodied in, a physical product […]
b) Convey the object code in, or embodied in, a physical product (including a physical distribution medium), accompanied by a written offer, valid for at least three years and valid for as long as you offer spare parts or customer support for that product model, to give anyone who possesses the object code either (1) a copy of the Corresponding Source for all the software in the product that is covered by this License[…]
c) Convey individual copies of the object code with a copy of the written offer to provide the Corresponding Source. This alternative is allowed only occasionally and noncommercially, and only if you received the object code with such an offer, in accord with subsection 6b.
[…]
I think there’s a reason R code is not compiled into apps much, e.g. few games in R… The GPL makes it very hard to hide the source (that’s its point), but by now also very hard to offer it for a limited time (with v3 that R itself changed to from v2). This is not a bug if you’re e.g. Richard Stallman (who started copyleft), then it’s a feature.
I think this is why R package code is distributed in source code form (sometimes also binary?), and compiled in your machine. Nobody dares to just provide binaries. It’s good for science, it should be open, and source always there, or at least available.
@kristoffer.carlsson
Right. In short, it’s not the responsibility of PackageCompiler, nor even desirable to bundle at least R (but would be desirable to at least have Python itself bundled sometimes; how, might be outside of the scope of PackageCompiler since it can be, or add as an option there?).
I explained in my last comment why even if possible to bundle R, it would a special sort of legal nightmare, so I at least think Kristoffer shouldn’t bother making it easier for users.
Actually the main licence (see COPYING) of R interpreter is GPLv2, not v3. Both are meant to not hide the code, but at least v2 allows the limited to 3 years written offer. IF you use anything from the standard library with GPLv3, or worse with AGPL, or those from any package (GPLv3 seems popular for packages), then I wouldn’t recommend distribuing R compiled package code (with or without PackageCompiler) unless it’s only for non-commercial.
Some even stay away from R because of GPLv2 (this, nor v3, should turn off scientists; not distributing compiled code):
Recently I wanted my company to build a product based on R. It simply seemed like a perfect fit.But this turned out to be a slippery slope into the open-source code licensing field, which I wasn’t really aware of before.
Bottom line: legal advice was not to use R!
Was it a single lawyer? No. The company was willing to “play along” with me, and we had a consultation with 4 different software lawyers, one after the other.
What is the issue? R is licensed as GPL 2, and most R packages are also GPL (whether 2 or 3).
The following licenses are in use for R or associated software such as packages.
The “GNU Affero General Public License” version 3
The “Artistic License” version 2.0
The “BSD 2-clause License”
The “BSD 3-clause License”
The “GNU General Public License” version 2
The “GNU General Public License” version 3
[…]
Some files are licensed under ‘GPL (version 2 or later)’, which includes GPL-3. See the comments in the files to see if this applies.
The AGPL is a fine free software licence (also for scientist), if you want the Affero-clause, stopping software-as-a-service (what Stallman calls software-as-a-service-substitute, SaaSS), but many companies are even more afraid of it then any either version of the GPL:
The GPL issue is interesting, but I will ignore it since it always ends up as a rabbit hole and you can only find out the truth for sure by going to court.
There will probably be multiple speed bumps with this I think, however, the first I will flag up is the slightly unusual and fragile way that RCall.jl keeps track of where the R installation is. You can choose where the R installation is at installation time by setting the R_HOME environment variable. I guess with PackageCompiler once you move your bundle to another machine, R_HOME will be pointing at the wrong location from the compiling machine.
I believe this design of RCall might have been decided because it helps with adding a REPL mode and the like. Maybe, possibly libR.so could be given to runtime if there was another RCallCore.jl package which could successfully be compiled without libR.so and then the higher level RCall.jl could depend upon this.
I believe this will not help with PackageCompiler, but I have made a this PR to make it possible to to configure libR a tiny bit later by blocking precompilation when no libR is available, and using Preferences.jl so precompilation is retriggered when a preference for the location of libR is changed. The reason I don’t think it will work with PackageCompiler, is as I understand it, all packages have to support precompilation for PackageCompiler. Or is it that we are still able to have some non-precompiled packages in our bundle with PackageCompiler? In the latter case, this should help, but you’ll still need to arrange for R to be installed, find it, and add its location to LocalPreferences.toml – e.g. with CondaPkg
using CondaPkg
using Preferences
using Libdl
using PreferenceTools
function locate_libR(Rhome)
@static if Sys.iswindows()
libR = joinpath(Rhome, "bin", Sys.WORD_SIZE==64 ? "x64" : "i386", "R.dll")
else
libR = joinpath(Rhome, "lib", "libR.$(Libdl.dlext)")
end
return libR
end
CondaPkg.resolve()
target_rhome = "$(CondaPkg.envdir())/lib/R"
PreferenceTools.add(
"RCall",
"Rhome" => target_rhome,
"libR" => locate_libR(target_rhome)
)
You would need to arrange for that (or something similar if you don’t want to use CondaPkg) to be run on the target machine (not the bundle creation machine) and in separate Julia session before you main bundle entrypoint is run.
I didn’t think RCall downloaded Julia for you, but I was mistaken, it does if you set R_HOME = “*”, using Conda. I’m not sure, that might still be at installation time, but since possible, could at least potentially be a runtime check.
About R packages themselves, I think they are usually downloaded as source and compiled, and arguably could also be done at runtime (Julia could have C++_compiler_JLL…, maybe does already, to provide all needed infrastructure, it’s just would you want to wait for compilation, and bundle it too or wait for it being downloaded, when first using?). I don’t know what package manager most R users use if any, but since R itself is provided by Conda, maybe at least some R packages are there also, precompiled?
I believe all (Julia) code can be supported (well one known exception, I forget the exact details), and my understanding is that there are no requirements.
Precompilation is on by default in packages (for a long time now), can be turned off (is it often done?), and I would have thought works either way. Because if PackageCompiler.jl wouldn’t (pre)compile all code (the main point), it still has LLVM at runtime and could compile. You can strip out LLVM, and then you would be more restricted. And since LLVM is big you might want to do that, but if you don’t care too much about non-small then not an issue. You will not get tiny binaries either way.
The GPL has been tested by a court, at least in Germany, and is being enforced in some casees (it’s never the GPL per se but for each specific GPL piece of code, e.g. busybox, and maybe nobody would bother for R or R packages). But if you don’t have an issue with the GPL/comply, e.g. provide a source with (for at least R packages), then I should be ok. Just note your full app will be “GPL as a whole”.
I didn’t think RCall downloaded Julia for you, but I was mistaken, it does if you set R_HOME = “*”, using Conda. I’m not sure, that might still be at installation time, but since possible, could at least potentially be a runtime check.
Yes it will do it at installation time with Conda.jl , and then hard-code the absolute path to libR.so into the RCall.jl source. This can sometimes cause problems locally, but will definitely cause problems when the RCall.jl is moved to another machine – which is often the case with PackageCompiler.jl.
About R packages themselves, I think they are usually downloaded as source and compiled, and arguably could also be done at runtime (Julia could have C++_compiler_JLL…, maybe does already, to provide all needed infrastructure, it’s just would you want to wait for compilation, and bundle it too or wait for it being downloaded, when first using?). I don’t know what package manager most R users use if any, but since R itself is provided by Conda, maybe at least some R packages are there also, precompiled?
R has built in package management with package.install(...), but then you need to manage when it’s run i.e. make sure it’s run exactly once. This is why I favor using Conda and particularly CondaPkg.jl. You can declare all your dependencies including R and the R libraries in a CondaPkg.toml and it will make sure everything is up to date every time you enter the environment (so at run time).
Because if PackageCompiler.jl wouldn’t (pre)compile all code (the main point), it still has LLVM at runtime and could compile.
Great news – in this case it may work — although in a somewhat rickety way with some things packaged and other things downloaded later on.