How to use package extensions

A while back, I thought I figured out how to use the new package extensions feature. Unfortunately, I was wrong. I tried putting together a working example here. I added it to a new environment along with Distributions.jl. When I run,

using PackageExtensionsExample
using Distributions

I get the following error:

[ Info: Precompiling DistributionsExt [e012062e-4c15-5e28-8fa8-ee6d8c117614]
ERROR: LoadError: ArgumentError: Package DistributionsExt does not have StatsBase in its dependencies:
- You may have a partially installed environment. Try `Pkg.instantiate()`
  to ensure all packages in the environment are installed.
- Or, if you have DistributionsExt checked out for development and have
  added StatsBase as a dependency but haven't updated your primary
  environment's manifest file, try `Pkg.resolve()`.
- Otherwise you may need to report an issue with DistributionsExt
 [1] macro expansion
   @ ./loading.jl:1634 [inlined]
 [2] macro expansion
   @ ./lock.jl:267 [inlined]
 [3] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:1611
 [4] include
   @ ./Base.jl:457 [inlined]
 [5] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::String)
   @ Base ./loading.jl:2049
 [6] top-level scope
   @ stdin:3
in expression starting at /home/dfish/.julia/packages/PackageExtensionsExample/MDoUL/ext/DistributionsExt.jl:1
in expression starting at stdin:3
β”Œ Error: Error during loading of extension DistributionsExt of PackageExtensionsExample, use `Base.retry_load_extensions()` to retry.
β”‚   exception =
β”‚    1-element ExceptionStack:
β”‚    Failed to precompile DistributionsExt [e012062e-4c15-5e28-8fa8-ee6d8c117614] to "/home/dfish/.julia/compiled/v1.9/DistributionsExt/jl_V6JDA5".
β”‚    Stacktrace:
β”‚      [1] error(s::String)
β”‚        @ Base ./error.jl:35
β”‚      [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool)
β”‚        @ Base ./loading.jl:2300
β”‚      [3] compilecache
β”‚        @ ./loading.jl:2167 [inlined]
β”‚      [4] _require(pkg::Base.PkgId, env::Nothing)
β”‚        @ Base ./loading.jl:1805
β”‚      [5] _require_prelocked(uuidkey::Base.PkgId, env::Nothing)
β”‚        @ Base ./loading.jl:1660
β”‚      [6] _require_prelocked(uuidkey::Base.PkgId)
β”‚        @ Base ./loading.jl:1658
β”‚      [7] run_extension_callbacks(extid::Base.ExtensionId)
β”‚        @ Base ./loading.jl:1255
β”‚      [8] run_extension_callbacks(pkgid::Base.PkgId)
β”‚        @ Base ./loading.jl:1290
β”‚      [9] run_package_callbacks(modkey::Base.PkgId)
β”‚        @ Base ./loading.jl:1124
β”‚     [10] _require_prelocked(uuidkey::Base.PkgId, env::String)
β”‚        @ Base ./loading.jl:1667
β”‚     [11] macro expansion
β”‚        @ ./loading.jl:1648 [inlined]
β”‚     [12] macro expansion
β”‚        @ ./lock.jl:267 [inlined]
β”‚     [13] require(into::Module, mod::Symbol)
β”‚        @ Base ./loading.jl:1611
β”‚     [14] eval
β”‚        @ ./boot.jl:370 [inlined]
β”‚     [15] include_string(mapexpr::typeof(REPL.softscope), mod::Module, code::String, filename::String)
β”‚        @ Base ./loading.jl:1903
β”‚     [16] invokelatest(::Any, ::Any, ::Vararg{Any}; kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
β”‚        @ Base ./essentials.jl:819
β”‚     [17] invokelatest(::Any, ::Any, ::Vararg{Any})
β”‚        @ Base ./essentials.jl:816
β”‚     [18] inlineeval(m::Module, code::String, code_line::Int64, code_column::Int64, file::String; softscope::Bool)
β”‚        @ VSCodeServer ~/.vscode/extensions/julialang.language-julia-1.54.2/scripts/packages/VSCodeServer/src/eval.jl:261
β”‚     [19] (::VSCodeServer.var"#67#72"{Bool, Bool, Bool, Module, String, Int64, Int64, String, VSCodeServer.ReplRunCodeRequestParams})()
β”‚        @ VSCodeServer ~/.vscode/extensions/julialang.language-julia-1.54.2/scripts/packages/VSCodeServer/src/eval.jl:181
β”‚     [20] withpath(f::VSCodeServer.var"#67#72"{Bool, Bool, Bool, Module, String, Int64, Int64, String, VSCodeServer.ReplRunCodeRequestParams}, path::String)
β”‚        @ VSCodeServer ~/.vscode/extensions/julialang.language-julia-1.54.2/scripts/packages/VSCodeServer/src/repl.jl:274
β”‚     [21] (::VSCodeServer.var"#66#71"{Bool, Bool, Bool, Module, String, Int64, Int64, String, VSCodeServer.ReplRunCodeRequestParams})()
β”‚        @ VSCodeServer ~/.vscode/extensions/julialang.language-julia-1.54.2/scripts/packages/VSCodeServer/src/eval.jl:179
β”‚     [22] hideprompt(f::VSCodeServer.var"#66#71"{Bool, Bool, Bool, Module, String, Int64, Int64, String, VSCodeServer.ReplRunCodeRequestParams})
β”‚        @ VSCodeServer ~/.vscode/extensions/julialang.language-julia-1.54.2/scripts/packages/VSCodeServer/src/repl.jl:38
β”‚     [23] (::VSCodeServer.var"#65#70"{Bool, Bool, Bool, Module, String, Int64, Int64, String, VSCodeServer.ReplRunCodeRequestParams})()
β”‚        @ VSCodeServer ~/.vscode/extensions/julialang.language-julia-1.54.2/scripts/packages/VSCodeServer/src/eval.jl:150
β”‚     [24] with_logstate(f::Function, logstate::Any)
β”‚        @ Base.CoreLogging ./logging.jl:514
β”‚     [25] with_logger
β”‚        @ ./logging.jl:626 [inlined]
β”‚     [26] (::VSCodeServer.var"#64#69"{VSCodeServer.ReplRunCodeRequestParams})()
β”‚        @ VSCodeServer ~/.vscode/extensions/julialang.language-julia-1.54.2/scripts/packages/VSCodeServer/src/eval.jl:253
β”‚     [27] #invokelatest#2
β”‚        @ ./essentials.jl:819 [inlined]
β”‚     [28] invokelatest(::Any)
β”‚        @ Base ./essentials.jl:816
β”‚     [29] macro expansion
β”‚        @ ~/.vscode/extensions/julialang.language-julia-1.54.2/scripts/packages/VSCodeServer/src/eval.jl:34 [inlined]
β”‚     [30] (::VSCodeServer.var"#62#63")()
β”‚        @ VSCodeServer ./task.jl:514
β”” @ Base loading.jl:1261

Resolve does not do anything. I’m probably doing something stupid, but I have no idea what it is.

I think that if you remove using StatsBase from the extension package and do using StatsBase in PackageExtensionsExample, it should work

Ah you also need to make StatsBase a dependency not a weak dependency!

Thanks for your reply. It looks like I need using StatsBase in both locations (see the updated commit).

I was hoping that I could use load StatsBase (or other package) only with the extension. This would be helpful in cases where the other dependency is large. Is there a work around?

The workaround I used in the past is to create an extension for each package you want to lazy import, and then stitch them together somehow, not a very good description but you could see this at work in the plotting extensions of Agents.jl for an example. Probably it is not always possible/easy to do so in all situations.

1 Like

Thanks for your help. It’s a bummer that it does not support extra dependencies easily. That greatly limits the scope of application.

1 Like

I totally agree, probably reporting an issue in Pkg.jl could be useful, if there isn’t really a better solution

Its now working without the using StatsBase statement in the main package module. When I tried that initially, it did not seem to work. Really weird, but this isn’t the first time that it seemed to work one way, and then it stopped working…

wait a second, that seems legit actually…the workaround I was referring to is for delaying also the download and precompilation of (in this case) the StatsBase package, but I think that if this is not a problem for you then you are good to go! In my case it was a problem since in general plotting packages are heavy to download&precompile and so I would have liked to delay it more simply

1 Like

Opened an issue about this: Dependencies of an Extension Β· Issue #3641 Β· JuliaLang/Pkg.jl Β· GitHub. Let’s see how it is received :slight_smile:

1 Like

Good idea. Thanks!

1 Like

I’d also be interested in whether others have inconsistent behavior with dependencies of an extension. At least its working for now :laughing:

1 Like