PackageCompiler throwing error on fresh machine

General Usage
,
1

I have been able to successfully use PackageCompiler to compile a Julia script into a binary and run on a different machine (same OS and distro) that doesn’t have Julia or any dependencies. For some reason, my workflow no longer seems to work as it used to. Here is what I’ve done:

$ wget https://julialang-s3.julialang.org/bin/linux/x64/1.6/julia-1.6.2-linux-x86_64.tar.gz
$ tar zxvf julia-1.6.2-linux-x86_64.tar.gz
$ julia
> ] add PkgTemplates PackageCompiler
> t = Template(;user="username")
> t("app")
> cd("/home/admin/.julia/dev/app") 
> ] activate
> ] dev .
> activate /home/admin/.julia/dev/app
> ] add DataFrames CSV Statistics LinearAlgebra DSP SingularSpectrumAnalysis Loess
> using Pkg; Pkg.precompile()
> exit()

Now to add artifacts file (not sure if this is necessary but it worked in the past? …

$ cd app
$ vim Artifacts.toml

Paste this into file:

[[fooifier]]
arch = "x86_64"
git-tree-sha1 = "98d93024ca384050c59d554415b75d61e467fd8c"
libc = "glibc"
os = "linux"

    [[fooifier.download]]
    sha256 = "5208c63a9d07e592c78f541fc13caa8cd191b11e7e77b31d407237c2b13ec391"
    url = "https://github.com/staticfloat/small_bin/raw/master/libfoo/libfoo.x86_64-linux-gnu.tar.gz"

[[fooifier]]
arch = "i686"
git-tree-sha1 = "c3a9f27382862092e064bcf4aeb3cb7190578338"
libc = "glibc"
os = "linux"

    [[fooifier.download]]
    sha256 = "97655b6a218d61284723b6923d7c96e6a256fa68b9419d723c588aa24404b102"
    url = "https://github.com/staticfloat/small_bin/raw/master/libfoo/libfoo.i686-linux-gnu.tar.gz"

[[fooifier]]
arch = "x86_64"
git-tree-sha1 = "f413ff2438a4e9e9dd69b23c35ca30de6af069cc"
os = "macos"

    [[fooifier.download]]
    sha256 = "fcc268772d6f21d65b45fcf3854a3142679b78e53c7673dac26c95d6ccc89a24"
    url = "https://github.com/staticfloat/small_bin/raw/master/libfoo/libfoo.x86_64-apple-darwin14.tar.gz"

[[fooifier]]
arch = "x86_64"
git-tree-sha1 = "d61f806c76b57e54f343634c5219d00d4c81b077" # FIX!
os = "windows"

    [[fooifier.download]]
    sha256 = "7f8939e9529835b83810d3ae7e2556f6e002d571f619894e54ece42ea5262b7f"
    url = "https://github.com/staticfloat/small_bin/raw/master/libfoo/libfoo.x86_64-w64-mingw32.tar.gz"

[[fooifier]]
arch = "aarch64"
git-tree-sha1 = "281cbe3dd65aa4bdb887bfb29651da500c81e242"
os = "linux"

    [[fooifier.download]]
    sha256 = "36886ac25cf5678c01fe20630b413f9354b7a3721c6a2c2043162f7ebd147ff5"
    url = "https://github.com/staticfloat/small_bin/raw/master/libfoo/libfoo.aarch64-linux-gnu.tar.gz"

Now add src code into src/app.jl

Then compile:

$ julia
> using PackageCompiler
> create_app("../app","app_compiled";force=true)

In the past, I’ve been able to transfer the output directory from this compilation (app_compiled) to a brand new machine with the same OS and distribution and it runs as expected. Now however, the process builds without error, but when I transfer the compiled binary, I get this error:

admin@ip-172-31-23-78:~$ app_compiled/bin/app input.csv output.csv
fatal: error thrown and no exception handler available.
InitError(mod=:FFTW, error=ErrorException("could not load library "/home/admin/.julia/artifacts/e8fff3113d692608a8bb8ab8492ec2f02a0de103/lib/libfftw3.so"
/home/admin/.julia/artifacts/e8fff3113d692608a8bb8ab8492ec2f02a0de103/lib/libfftw3.so: cannot open shared object file: No such file or directory"))
jl_errorf at /buildworker/worker/package_linux64/build/src/rtutils.c:77
jl_load_dynamic_library at /buildworker/worker/package_linux64/build/src/dlload.c:266
jl_get_library_ at /buildworker/worker/package_linux64/build/src/runtime_ccall.cpp:48
jl_get_library_ at /buildworker/worker/package_linux64/build/src/runtime_ccall.cpp:32 [inlined]
jl_lazy_load_and_lookup at /buildworker/worker/package_linux64/build/src/runtime_ccall.cpp:80
fftw_init_threads at /home/admin/.julia/packages/FFTW/kKdEk/src/providers.jl:69 [inlined]
__init__ at /home/admin/.julia/packages/FFTW/kKdEk/src/FFTW.jl:30
jfptr___init___35047 at /home/admin/ app_compiled/bin/app.so (unknown line)
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2237 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2419
jl_apply at /buildworker/worker/package_linux64/build/src/julia.h:1703 [inlined]
jl_module_run_initializer at /buildworker/worker/package_linux64/build/src/toplevel.c:72
_julia_init at /buildworker/worker/package_linux64/build/src/init.c:794
main at  app_compiled/bin/app (unknown line)
__libc_start_main at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
_start at  app_compiled/bin/app (unknown line)

If useful, I am using Julia 1.6.2 and Debian 10.

2

What FFTW version is this?

1 Like
3

I’m not sure, I don’t even specifically add FFTW? Will look, maybe its a dependency on one of my packages.

On the machine I am doing the compiling:

julia> using FFTW
ERROR: ArgumentError: Package FFTW not found in current path:
- Run `import Pkg; Pkg.add("FFTW")` to install the FFTW package.

Stacktrace:
 [1] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:893

I’ve never needed to explicitly add FFTW during compilation, do you think this would help?

4

FFTW is what is erroring though. You should be able to see its version by doing pkg> st -m.

1 Like
5

Ah, okay.

I ran the command in my base environment:

  [7a1cc6ca] FFTW v1.4.3

I never explicitly downloaded FFTW, so it must be native when installing Julia?

6

No, one of the packages you have added depends on FFTW, so it gets pulled down automatically. I think this should be fixed in FFTW 1.4.4 via the fix in make FFTW relocatable, take 2 by KristofferC · Pull Request #212 · JuliaMath/FFTW.jl · GitHub.

2 Likes
7

Ah it worked! I just added FFTW as a part of the package add line in my workflow, which updated to 1.4.5. Thanks for the tip.