Minimising top-level project latency

General Usage Performance
1

If I want to use a third-party package with minimal latency overhead, I can do so by having it precompiled. However, I am in the top-level project in a codebase that implements a complex pipeline and get this error when instantiateing:

(MyPackage) pkg> instantiate
Precompiling project...
  Progress [========================================>]  1/1
  ✗ MyPackage
0 dependencies successfully precompiled in 3 seconds (102 already precompiled)
1 dependency errored

Is there any way I can debug this error? Bear in mind this project is user-facing, so I am not sure if it can be precompiled, or if only some parts can be precompiled.

2

AFAIK you should get an actual error message if you call

] precompile

explicitly.
To be safe, I would try it in a fresh REPL.
If the error you get does not occur in your code but in some dependency, try updating your environment. In the worst case you might have to wait for an update of your dependencies I think.

Generally speaking, precompilation should work. Even less overhead is achieved by using PackageCompiler.jl, but you should only attempt this if your package precompiled successfully and you don’t have to make any more changes to it.

1 Like
3

So the command the package uses is this:

julia --project src/MyPackage.jl some-arg-goes-here

And that file looks like this:

using Compat

#  Verify if installed Julia version is in range of required Julia versions
minver = VersionNumber("1.6.0")
if VERSION < minver
    @error "Your Julia version is $(VERSION) while MyPackage requires Julia version to be a minimum $(minver)"
    exit(1)
end

include("AllModules.jl")  # I am not defending this design choice

import .AllModules

AllModules.Pipeline.run(Base.ARGS)

The issue is that when I try to use precompile I get this error trace, and I definitely don’t understand this. Happens with or without using Compat above:

(MyPackage) pkg> precompile
ERROR: LoadError: UndefVarError: include not defined
Stacktrace:
 [1] top-level scope
   @ ~/Develop/mypackage/src/MyPackage.jl:10
 [2] include
   @ ./Base.jl:386 [inlined]
 [3] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::Nothing)
   @ Base ./loading.jl:1213
 [4] top-level scope
   @ none:1
 [5] eval
   @ ./boot.jl:360 [inlined]
 [6] eval(x::Expr)
   @ Base.MainInclude ./client.jl:446
 [7] top-level scope
   @ none:1

Not sure if it has something to do with this issue?

4

What is the reason why you select src/MyPackage.jl as the directory of your environment? Personally, I tend to activate environments in the REPL, but as I understand it, you should put the directory of the Project.toml file here (usually in the parent directory of src/. This is probably not your issue though.

The issue points to the PackageCompiler package, I assume you are not using this, correct?

Which of these things are you trying to do?

  1. Create a package, which is then precompiled upon its first use?
  2. Or use PackageCompiler.jl to compile a script. The script will be run and the fully compiled functions are saved to a file which can be used to start later sessions.

If the answer is 1.:
Possibly the reason why you cannot precompile is that your package is not defined as a module? Usually, a Julia package looks like this:

module MyPackage
using Compat

#  Verify if installed Julia version is in range of required Julia versions
minver = VersionNumber("1.6.0")
if VERSION < minver
    @error "Your Julia version is $(VERSION) while MyPackage requires Julia version to be a minimum $(minver)"
    exit(1)
end

include("AllModules.jl")  # I am not defending this design choice

import .AllModules

AllModules.Pipeline.run(Base.ARGS)
end #Module

Does the error go away if you define your package this way and include it with

]activate MyPackage
using MyPackage

If the answer is 2. Try first to run your script with

julia src/MyPackage.jl

if this runs without errors, you can try to follow the steps given in

to create a sysimage.
Hope this helps