How to use the PackageCompiler with a private registry

General Usage
1

I have the following structure:

├──App/
├──compiled/
│    ├── make.jl
│    ├── precompile.jl
├──deps/
├──Project.toml

where make.jl is

using PackageCompiler

PackageCompiler.create_app(
    "App",
    "compiled/App";
    cpu_target="generic",
    sysimage_build_args=`-O3`,
    precompile_execution_file="compiled/precompile.jl",
    include_transitive_dependencies=false,
    include_lazy_artifacts=true,
    incremental=true,
)

and precompile.jl runs some exemplary workload

import App
import JSON

App.run()
# something else

In my root Project.toml i have only PackageCompiler.jl as dependency.
And in my App I have some dependencies, including local modules via [sources] and packages from a private registry.

When I run include("compiled/make.jl") I get an error message that one of the private packages is not registered. ERROR: expected package .... to be registered

In my App I also have tests which run fine.

1 Like
2

In Pkg.Registry.reachable_registries() I see my registry.

And with debug enabled I see:

...
  Total artifact file size: 35.037 MiB
┌ Debug: instantiating project at "C:\\<path-to-project>\\App"
└ @ PackageCompiler C:\JuliaPackages\packages\PackageCompiler\cTtGY\src\PackageCompiler.jl:582
┌ Debug: ensurecompiled: running `'C:\Julia_newest\bin\julia.exe' --color=yes --startup-file=no --pkgimages=no '--sysimage=C:\Julia_newest\lib\julia\sys.dll' -e 'using Pkg; Pkg.precompile()'`
│   JULIA_LOAD_PATH = "C:\\<path-to-project>\\App;@stdlib"
└ @ PackageCompiler C:\JuliaPackages\packages\PackageCompiler\cTtGY\src\PackageCompiler.jl:298
    Updating registry at `C:\<user-path>\.julia\registries\General.toml`
ERROR: expected package `MyPackage [6c6c7b4e]` to be registered
3

I would have expected a line Updating registry at... for your private registry as well, so it doesn’t look like PackageCompiler is seeing the same registries as you are.

I’m regularly using PackageCompiler with a private registry at work without problems, but a difference is that we run our own package server, which makes Pkg install our private registry by default.

I have tried to disable the package server in order to reproduce this problem, without success, but I can’t say with confidence that I managed to set up a correct environment.

4

That is interesting.
Could you elaborate on how you do that?
I guess you are using this: GitHub - GunnarFarneback/LocalPackageServer.jl: Julia storage and package server for local packages.

5

I tried with replacing the get_julia_cmd method to include a startup file and installing the registry.

function PackageCompiler.get_julia_cmd()
    julia_path = joinpath(Sys.BINDIR, Base.julia_exename())
    color = if Base.have_color === nothing
        "auto"
    elseif Base.have_color
        "yes"
    else
        "no"
    end
    if isdefined(Base, :Linking) # pkgimage support feature flag
        `$julia_path --color=$color --startup-file=yes --pkgimages=no`
    else
        `$julia_path --color=$color --startup-file=yes`
    end
end

But I get now a different error:

Cloning registry from "<url-to-my-registry>MyRegistry.jl.git"
Registry `MyRegistry` already exists in `C:\<path>\registries\MyRegistry`.
ERROR: Precompiling failed to find source of parent package: "MyPackage" project...
6

I also tried to locally clone and dev the package. That works, however is not really feasible, since versioning does not work or is at least cumbersome.

7

Just checking for an unlikely mismatch, what does Pkg.Registry.status() say in the same scope where you would evaluate include("compiled/make.jl")?

8

Spot on.