Compile_package fails on nonregistered package

General Usage
1

I am trying to use compile_package or compile_incremental from PackageCompiler.jl to compile an internal package that is not registered in METADATA. When trying compile_increemental, the process fails with

[ Info: used 227 out of 227 precompile statements
┌ Info: activating new environment at ~/.julia/packages/PackageCompiler/oT98U/packages/Project.toml.
└ @ Pkg.API /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/Pkg/src/API.jl:519
  Updating registry at `~/.julia/registries/General`
  Updating git-repo `https://github.com/JuliaRegistries/General.git`
ERROR: LoadError: LoadError: ArgumentError: Package Robotlab [8223e05c-3ffc-11e9-04f8-dd73b465bce1] is required but does not seem to be installed:
 - Run `Pkg.instantiate()` to install all recorded dependencies.

Stacktrace:
 [1] _require(::Base.PkgId) at ./loading.jl:929
 [2] require(::Base.PkgId) at ./loading.jl:858
 [3] require(::Module, ::Symbol) at ./loading.jl:853
 [4] include at ./boot.jl:326 [inlined]
 [5] include_relative(::Module, ::String) at ./loading.jl:1038
 [6] include(::Module, ::String) at ./sysimg.jl:29
 [7] top-level scope at none:0
in expression starting at /local/home/fredrikb/.julia/packages/PackageCompiler/oT98U/packages/incremental_precompile.jl:17
in expression starting at /local/home/fredrikb/.julia/packages/PackageCompiler/oT98U/sysimg/run_julia_code.jl:10

and when trying compile_package, the process fails with the same error and a segmentation fault at that. I can’t find any related issues in the repo for PackageCompiler. Is compilation of a non-registered package not supported or have I run into a bug?

The package in question (Robotlab.jl) passes tests and is usable locally.
Setup:

julia> versioninfo()
Julia Version 1.1.0
Commit 80516ca202 (2019-01-21 21:24 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Core(TM) i7-4790 CPU @ 3.60GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, haswell)
Environment:
  JULIA_NUM_THREADS = 4
  JULIA_SVG_BROWSER = Google Chrome.app

(v1.1) pkg> st
    Status `~/.julia/environments/v1.1/Project.toml`
  [9b87118b] PackageCompiler v0.6.3
  [d3a5d4b2] Robotlab v0.0.0 [`~/.julia/dev/Robotlab`]

Edit:
I just noticed that the last release is called fix for unregistered packages, unfortunately, it did not seem to fix the issue in my case :open_mouth:

1 Like
2

I think you need to push the path to your local package to Base.LOAD_PATH before calling compile_incremental (see here).

Should presumably be documented somewhere, so do open issues/PRs! :wink:

3

Unfortunately, that does not solve the problem, after having pushed the following to LOAD_PATH, the same error persists

julia> LOAD_PATH
6-element Array{String,1}:
 "@"                                            
 "@v#.#"                                        
 "@stdlib"                                      
 "/local/home/fredrikb/.julia/dev/Robotlab/"    
 "/local/home/fredrikb/.julia/dev/"             
 "/local/home/fredrikb/.julia/dev/Robotlab/src/"
4

Have same issue:

julia> using PackageCompiler

julia> PackageCompiler.compile_incremental(:test123, verbose = true, force = false)
Updating registry at c:\space\jdepot\registries\General
Updating git-repo https://github.com/JuliaRegistries/General.git
Resolving package versions…
Updating C:\space\jdepot\dev\PackageCompiler\packages\test123\Project.toml
[de0858da] + Printf
Updating C:\space\jdepot\dev\PackageCompiler\packages\test123\Manifest.toml
[de0858da] + Printf
[4ec0a83e] + Unicode
Resolving package versions…
Updating C:\space\jdepot\dev\PackageCompiler\packages\test123\Project.toml
[9b87118b] + PackageCompiler v0.6.4
[44cfe95a] + Pkg
Updating C:\space\jdepot\dev\PackageCompiler\packages\test123\Manifest.toml
[9e28174c] + BinDeps v0.8.10
[b99e7846] + BinaryProvider v0.5.7
[e1450e63] + BufferedStreams v1.0.0
[34da2185] + Compat v2.2.0
[0862f596] + HTTPClient v0.2.1
[b27032c2] + LibCURL v0.5.2
[522f3ed2] + LibExpat v0.5.0
[2ec943e9] + Libz v1.0.0
[9b87118b] + PackageCompiler v0.6.4
[30578b45] + URIParser v0.4.0
[c17dfb99] + WinRPM v0.4.2
[2a0f44e3] + Base64
[ade2ca70] + Dates
[8bb1440f] + DelimitedFiles
[8ba89e20] + Distributed
[b77e0a4c] + InteractiveUtils
[76f85450] + LibGit2
[8f399da3] + Libdl
[37e2e46d] + LinearAlgebra
[56ddb016] + Logging
[d6f4376e] + Markdown
[a63ad114] + Mmap
[44cfe95a] + Pkg
[3fa0cd96] + REPL
[9a3f8284] + Random
[ea8e919c] + SHA
[9e88b42a] + Serialization
[1a1011a3] + SharedArrays
[6462fe0b] + Sockets
[2f01184e] + SparseArrays
[10745b16] + Statistics
[8dfed614] + Test
[cf7118a7] + UUIDs
Activating environment at c:\space\jdepot\dev\PackageCompiler\packages\test123\Project.toml
ERROR: LoadError: Package test123 [fb82510b-2530-40ac-8f12-c46bb3d5f6d3] not found in a registry.
Stacktrace:
[1] pkgerror(::String) at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.2\Pkg\src\Types.jl:112
[2] check_registered(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.2\Pkg\src\Operations.jl:924
[3] up(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}, ::Pkg.Types.UpgradeLevel) at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.2\Pkg\src\Operations.jl:1043
[4] #up#43(::Pkg.Types.UpgradeLevel, ::Pkg.Types.PackageMode, ::Bool, ::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(Pkg.API.up), ::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.2\Pkg\src\API.jl:167
[5] #up#38 at .\none:0 [inlined]
[6] #up at .\none:0 [inlined]
[7] resolve at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.2\Pkg\src\API.jl:172 [inlined] (repeats 2 times)
[8] top-level scope at c:\space\jdepot\dev\PackageCompiler\sysimg\run_julia_code.jl:6
[9] include at .\boot.jl:328 [inlined]
[10] include_relative(::Module, ::String) at .\loading.jl:1094
[11] include(::Module, ::String) at .\Base.jl:31
[12] exec_options(::Base.JLOptions) at .\client.jl:295
[13] _start() at .\client.jl:464
in expression starting at c:\space\jdepot\dev\PackageCompiler\sysimg\run_julia_code.jl:6
ERROR: failed process: Process('C:\space\julia\j120\bin\julia.exe' --compile=all --optimize=0 -g1 '--trace-compile=c:\space\jdepot\dev\PackageCompiler\packages\precompile_tmp.jl' --startup-file=no --history-file=yes --code-coverage=none --inline=yes --math-mode=ieee --handle-signals=yes --warn-overwrite=no --compile=yes --depwarn=yes --cpu-target=native --track-allocation=none --sysimage-native-code=yes '--sysimage=C:\space\julia\j120\lib\julia\sys.dll' -g1 --compiled-modules=yes --optimize=2 'c:\space\jdepot\dev\PackageCompiler\sysimg\run_julia_code.jl', ProcessExited(1)) [1]

Stacktrace:
[1] pipeline_error at .\process.jl:813 [inlined]
[2] #run#536(::Bool, ::typeof(run), ::Cmd) at .\process.jl:728
[3] run at .\process.jl:726 [inlined]
[4] #run_julia#1 at c:\space\jdepot\dev\PackageCompiler\src\compiler_flags.jl:225 [inlined]
[5] #run_julia at .\none:0 [inlined]
[6] snoop(::Symbol, ::String, ::String, ::String, ::Bool, ::Array{Any,1}) at c:\space\jdepot\dev\PackageCompiler\src\snooping.jl:34
[7] (::getfield(PackageCompiler, Symbol(“##35#37”)){Array{Any,1},Tuple{Symbol},Dict{Any,Any},String,Dict{String,Array{Dict{String,Any},1}}})(::IOStream) at c:\space\jdepot\dev\PackageCompiler\src\snooping.jl:125
[8] #open#312(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(open), ::getfield(PackageCompiler, Symbol(“##35#37”)){Array{Any,1},Tuple{Symbol},Dict{Any,Any},String,Dict{String,Array{Dict{String,Any},1}}}, ::String, ::Vararg{String,N} where N) at .\iostream.jl:375
[9] open at .\iostream.jl:373 [inlined]
[10] #snoop_packages#34 at c:\space\jdepot\dev\PackageCompiler\src\snooping.jl:110 [inlined]
[11] #snoop_packages at .\none:0 [inlined]
[12] #compile_incremental#63 at c:\space\jdepot\dev\PackageCompiler\src\incremental.jl:176 [inlined]
[13] (::getfield(PackageCompiler, Symbol(“#kw##compile_incremental”)))(::NamedTuple{(:verbose, :force),Tuple{Bool,Bool}}, ::typeof(compile_incremental), ::Symbol) at .\none:0
[14] top-level scope at REPL[2]:1

5

Is there any update on this? I’m facing a similar error.

ERROR: LoadError: Package myPackage [16ed6a40-d4c3-11e9-1e10-59384530c1b4] not found in a registry.
Stacktrace:
[1] pkgerror(::String) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.3\Pkg\src\Types.jl:113
[2] check_registered(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.3\Pkg\src\Operations.jl:971
[3] up(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}, ::Pkg.Types.UpgradeLevel) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.3\Pkg\src\Operations.jl:1098
[4] #up#43(::Pkg.Types.UpgradeLevel, ::Pkg.Types.PackageMode, ::Bool, ::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(Pkg.API.up), ::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.3\Pkg\src\API.jl:170
[5] #up#38 at .\none:0 [inlined]
[6] #up at .\none:0 [inlined]
[7] resolve at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.3\Pkg\src\API.jl:175 [inlined] (repeats 2 times)
[8] top-level scope at C:\Users\lgo.julia\dev\PackageCompiler\sysimg\run_julia_code.jl:6
[9] include at .\boot.jl:328 [inlined]
[10] include_relative(::Module, ::String) at .\loading.jl:1105
[11] include(::Module, ::String) at .\Base.jl:31
[12] exec_options(::Base.JLOptions) at .\client.jl:287
[13] _start() at .\client.jl:460
in expression starting at C:\Users\lgo.julia\dev\PackageCompiler\sysimg\run_julia_code.jl:6

6

I have the same problem when using PackageCompiler v0.6.4, PackageCompiler’s master branch or any of the latest commits building on top of master.

I have, however, compiled successfully my non registered packages using PackageCompiler’s sd-notomls branch. You might want to try that instead. Be aware that this branch has other problems, as it errors on compiling the DataFrames package for example, see #290.