If I understand correctly this comment, the cpu_target="x86_64" option is not compatible when julia is “self built”, which is indeed the case on my workstation (I compiled it locally).
However, I’m also trying to compile it in a debian-based singularity container, that has a “shipped julia” installed. The goal is to be able to run the application “anywhere” (where “anywhere” currently means “on a computing cluster with no julia installation” and where the user should not need to bother with anything julia related, just loading singularity and running the container should work). For this, I need to cpu_target="x86_64" to be set.
Is there a way (either from the julia build script itself or from the OS) to determine whether julia is “self-build” or “shipped”, in order to adapt the compiling option automatically, without having to manually edit the script whenever the host system changes?
I hope I am not getting this totally wrong…
On your workstation you are building Julia from source.
However in the container build you should be pulling Julia won as a ‘shipped’ piece of software from the Julialang site. Am I understanding this correctly?
Thanks for your suggestion, but there is actually no issue with how to get julia in the container. I’m using a container that already has julia in it: Docker
The julia version in the container is compatible with the cpu_target="x86_64" option of PackageCompiler, and that’s what I actually need.
But for debugging purposes, I want to be able to use PackageCompiler on my workstation. The build is done via deps/build.jl, which looks as follows:
import Pkg
println("Building qaf_demux")
Pkg.add("REPL")
Pkg.add("PackageCompiler")
using PackageCompiler
# setting cpu_target does not work with a self-built Julia: https://github.com/NHDaly/ApplicationBuilder.jl/issues/62#issuecomment-503721859
# build_executable(joinpath(@__DIR__, "../bin/qaf_demux_to_compile.jl"), "qaf_demux", snoopfile=joinpath(@__DIR__, "../bin/snoop.jl"))
build_executable(joinpath(@__DIR__, "../bin/qaf_demux_to_compile.jl"), "qaf_demux", snoopfile=joinpath(@__DIR__, "../bin/snoop.jl"), cpu_target="x86_64")
This is all under version control, and whenever I want to try to build the application, I need to manually change which line is commented and which is not at the end of the above script. This in turn makes me see a modified file under git, but this is just for local use and is not meant to be committed.
What I would like is a way to have a build.jl that works both on my local workstation and in the container, at build time.
Something like that:
import Pkg
println("Building qaf_demux")
Pkg.add("REPL")
Pkg.add("PackageCompiler")
using PackageCompiler
# setting cpu_target does not work with a self-built Julia: https://github.com/NHDaly/ApplicationBuilder.jl/issues/62#issuecomment-503721859
if (the julia currently running this code is locally compiled)
build_executable(joinpath(@__DIR__, "../bin/qaf_demux_to_compile.jl"), "qaf_demux", snoopfile=joinpath(@__DIR__, "../bin/snoop.jl"))
else
build_executable(joinpath(@__DIR__, "../bin/qaf_demux_to_compile.jl"), "qaf_demux", snoopfile=joinpath(@__DIR__, "../bin/snoop.jl"), cpu_target="x86_64")
end
Maybe I should explicit what is meant by “shipped” here. This is the vocabulary I took from the linked comment in my original post:
Ah, sorry, it’s because --cpu-target=x86-64 is incompatible with self-built julia. You can either:
Use a shipped julia binary, or
Change the cpu target to cpu_target="native" in build_app_bundle(). But If you do this, then your app will be significantly less portable to other machines (it will only run on other machines with identical cpus).
Why don’t you use the same target as julia itself uses when building releases which is generic;sandybridge,-xsaveopt,clone_all;haswell,-rdrnd,base(1)? Julia can load optimized code based on runtime cpu feature detection but the generic fallback should allow it to work on most cpus.
These options work when building the application in the container (are they supposed to be “better” than a plain and general x86_64, but still very portable ?), but not on the local workstation. I get the very weird error again:
I then get another error: LoadError: MethodError: no method matching chosetranscoder(::SubString{String})
This too can probably fixed by changing type signatures of my functions, but how come this cpu-target compiling option interacts with allowed type signatures in my code?