How to use package extensions

General Usage
1

A while back, I thought I figured out how to use the new package extensions feature. Unfortunately, I was wrong. I tried putting together a working example here. I added it to a new environment along with Distributions.jl. When I run,

using PackageExtensionsExample
using Distributions

I get the following error:

[ Info: Precompiling DistributionsExt [e012062e-4c15-5e28-8fa8-ee6d8c117614]
ERROR: LoadError: ArgumentError: Package DistributionsExt does not have StatsBase in its dependencies:
- You may have a partially installed environment. Try `Pkg.instantiate()`
  to ensure all packages in the environment are installed.
- Or, if you have DistributionsExt checked out for development and have
  added StatsBase as a dependency but haven't updated your primary
  environment's manifest file, try `Pkg.resolve()`.
- Otherwise you may need to report an issue with DistributionsExt
Stacktrace:
 [1] macro expansion
   @ ./loading.jl:1634 [inlined]
 [2] macro expansion
   @ ./lock.jl:267 [inlined]
 [3] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:1611
 [4] include
   @ ./Base.jl:457 [inlined]
 [5] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::String)
   @ Base ./loading.jl:2049
 [6] top-level scope
   @ stdin:3
in expression starting at /home/dfish/.julia/packages/PackageExtensionsExample/MDoUL/ext/DistributionsExt.jl:1
in expression starting at stdin:3
┌ Error: Error during loading of extension DistributionsExt of PackageExtensionsExample, use `Base.retry_load_extensions()` to retry.
│   exception =
│    1-element ExceptionStack:
│    Failed to precompile DistributionsExt [e012062e-4c15-5e28-8fa8-ee6d8c117614] to "/home/dfish/.julia/compiled/v1.9/DistributionsExt/jl_V6JDA5".
│    Stacktrace:
│      [1] error(s::String)
│        @ Base ./error.jl:35
│      [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool)
│        @ Base ./loading.jl:2300
│      [3] compilecache
│        @ ./loading.jl:2167 [inlined]
│      [4] _require(pkg::Base.PkgId, env::Nothing)
│        @ Base ./loading.jl:1805
│      [5] _require_prelocked(uuidkey::Base.PkgId, env::Nothing)
│        @ Base ./loading.jl:1660
│      [6] _require_prelocked(uuidkey::Base.PkgId)
│        @ Base ./loading.jl:1658
│      [7] run_extension_callbacks(extid::Base.ExtensionId)
│        @ Base ./loading.jl:1255
│      [8] run_extension_callbacks(pkgid::Base.PkgId)
│        @ Base ./loading.jl:1290
│      [9] run_package_callbacks(modkey::Base.PkgId)
│        @ Base ./loading.jl:1124
│     [10] _require_prelocked(uuidkey::Base.PkgId, env::String)
│        @ Base ./loading.jl:1667
│     [11] macro expansion
│        @ ./loading.jl:1648 [inlined]
│     [12] macro expansion
│        @ ./lock.jl:267 [inlined]
│     [13] require(into::Module, mod::Symbol)
│        @ Base ./loading.jl:1611
│     [14] eval
│        @ ./boot.jl:370 [inlined]
│     [15] include_string(mapexpr::typeof(REPL.softscope), mod::Module, code::String, filename::String)
│        @ Base ./loading.jl:1903
│     [16] invokelatest(::Any, ::Any, ::Vararg{Any}; kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
│        @ Base ./essentials.jl:819
│     [17] invokelatest(::Any, ::Any, ::Vararg{Any})
│        @ Base ./essentials.jl:816
│     [18] inlineeval(m::Module, code::String, code_line::Int64, code_column::Int64, file::String; softscope::Bool)
│        @ VSCodeServer ~/.vscode/extensions/julialang.language-julia-1.54.2/scripts/packages/VSCodeServer/src/eval.jl:261
│     [19] (::VSCodeServer.var"#67#72"{Bool, Bool, Bool, Module, String, Int64, Int64, String, VSCodeServer.ReplRunCodeRequestParams})()
│        @ VSCodeServer ~/.vscode/extensions/julialang.language-julia-1.54.2/scripts/packages/VSCodeServer/src/eval.jl:181
│     [20] withpath(f::VSCodeServer.var"#67#72"{Bool, Bool, Bool, Module, String, Int64, Int64, String, VSCodeServer.ReplRunCodeRequestParams}, path::String)
│        @ VSCodeServer ~/.vscode/extensions/julialang.language-julia-1.54.2/scripts/packages/VSCodeServer/src/repl.jl:274
│     [21] (::VSCodeServer.var"#66#71"{Bool, Bool, Bool, Module, String, Int64, Int64, String, VSCodeServer.ReplRunCodeRequestParams})()
│        @ VSCodeServer ~/.vscode/extensions/julialang.language-julia-1.54.2/scripts/packages/VSCodeServer/src/eval.jl:179
│     [22] hideprompt(f::VSCodeServer.var"#66#71"{Bool, Bool, Bool, Module, String, Int64, Int64, String, VSCodeServer.ReplRunCodeRequestParams})
│        @ VSCodeServer ~/.vscode/extensions/julialang.language-julia-1.54.2/scripts/packages/VSCodeServer/src/repl.jl:38
│     [23] (::VSCodeServer.var"#65#70"{Bool, Bool, Bool, Module, String, Int64, Int64, String, VSCodeServer.ReplRunCodeRequestParams})()
│        @ VSCodeServer ~/.vscode/extensions/julialang.language-julia-1.54.2/scripts/packages/VSCodeServer/src/eval.jl:150
│     [24] with_logstate(f::Function, logstate::Any)
│        @ Base.CoreLogging ./logging.jl:514
│     [25] with_logger
│        @ ./logging.jl:626 [inlined]
│     [26] (::VSCodeServer.var"#64#69"{VSCodeServer.ReplRunCodeRequestParams})()
│        @ VSCodeServer ~/.vscode/extensions/julialang.language-julia-1.54.2/scripts/packages/VSCodeServer/src/eval.jl:253
│     [27] #invokelatest#2
│        @ ./essentials.jl:819 [inlined]
│     [28] invokelatest(::Any)
│        @ Base ./essentials.jl:816
│     [29] macro expansion
│        @ ~/.vscode/extensions/julialang.language-julia-1.54.2/scripts/packages/VSCodeServer/src/eval.jl:34 [inlined]
│     [30] (::VSCodeServer.var"#62#63")()
│        @ VSCodeServer ./task.jl:514
└ @ Base loading.jl:1261

Resolve does not do anything. I’m probably doing something stupid, but I have no idea what it is.

2

I think that if you remove using StatsBase from the extension package and do using StatsBase in PackageExtensionsExample, it should work

3

Ah you also need to make StatsBase a dependency not a weak dependency!

4

Thanks for your reply. It looks like I need using StatsBase in both locations (see the updated commit).

I was hoping that I could use load StatsBase (or other package) only with the extension. This would be helpful in cases where the other dependency is large. Is there a work around?

5

The workaround I used in the past is to create an extension for each package you want to lazy import, and then stitch them together somehow, not a very good description but you could see this at work in the plotting extensions of Agents.jl for an example. Probably it is not always possible/easy to do so in all situations.

6

Thanks for your help. It’s a bummer that it does not support extra dependencies easily. That greatly limits the scope of application.

7

I totally agree, probably reporting an issue in Pkg.jl could be useful, if there isn’t really a better solution

8

Its now working without the using StatsBase statement in the main package module. When I tried that initially, it did not seem to work. Really weird, but this isn’t the first time that it seemed to work one way, and then it stopped working…

9

wait a second, that seems legit actually…the workaround I was referring to is for delaying also the download and precompilation of (in this case) the StatsBase package, but I think that if this is not a problem for you then you are good to go! In my case it was a problem since in general plotting packages are heavy to download&precompile and so I would have liked to delay it more simply

10

Opened an issue about this: Dependencies of an Extension · Issue #3641 · JuliaLang/Pkg.jl · GitHub. Let’s see how it is received :slight_smile:

11

Good idea. Thanks!

12

I’d also be interested in whether others have inconsistent behavior with dependencies of an extension. At least its working for now :laughing: