I get that its intended use is to provide Requires.jl-like functionality, i.e. to run some code when some other package is loaded, e.g. in order to add compatibility methods.
Two questions:
Is there a mechanism to import the module defined in an extension package?
Is there a mechanism to install the packages for an extension? E.g. if package Foo had an extension FooExt depending on ExtPkg say, can I specify to install FooExt somehow (which implicitly installs ExtPkg) instead of specifying ExtPkg explicitly?
If not, are these on the roadmap?
The second suggestion is a lot like Python’s “extras”, which you’d install like
It appears that these modules do not have UUIDs so they may not be bonafide packages in themselves. The example prototypes show the extensions are usually modules which are included as submodules of the base package.
You can get the module using get_extension. Using PGFPlotsX as an example:
julia> using PGFPlotsX
julia> Base.get_extension(PGFPlotsX, :ColorsExt)
julia> using Colors
julia> Base.get_extension(PGFPlotsX, :ColorsExt)
ColorsExt
So before the extension is loaded get_extension will return nothing.
In general though, you should probably try to make getting the module not required to use the package. It is better to try have the extension add methods via dispatch and make the new functionality available in that way.
They get their UUID by combining the UUID of the parent package and the name of the extension module:
# Get UUID of extension module
julia> Base.PkgId(Base.get_extension(PGFPlotsX, :ColorsExt)).uuid
UUID("283d1826-985b-5544-82b8-7fd9aa83b823")
# Which is computed like this:
julia> Base.uuid5(Base.identify_package("PGFPlotsX").uuid, "ColorsExt")
UUID("283d1826-985b-5544-82b8-7fd9aa83b823")
I get your point for the intended usage of extensions, I was more wondering if the same mechanism could be co-opted to provide something a bit like Python’s “extras”.
In a way I’m asking if it’s possible to have multiple modules in a single Julia package which can be independently imported and have separate dependencies. The answer to that is seemingly currently “no” - which is fine because you can always create multiple packages instead.
Well yes, but I do think that best practices should be to make the functionality exposed by overloading functions in the parent. It can even be a zero method function that is overloaded. So you would not need to access the namespace of the extension.
I’m not sure whether this is answered in the above posts. I understand the question is about whether there is a possibility to export/import variables from the extension module.
In Reqires.jl it is possible to export variables. They are part of the parent module. For the extension module I have not been able to export or import variables, neither as part of the extension module, nor via @eval <parentmodule> export x, y as this would break precompilation.
From what I understand, it is not considered to be a good practice to allow additional objects to be exported conditionally. For extensions, this kind of export behavior is not allowed at all.
I’d like to learn more about this philosophy.
I think, it makes totally sense to have certain types available only if two (or more) packages are loaded.
I agree that they shouldn’t become part of the the parent module as it is the case with Require.jl, but leaving them in the extension module and allowing import/export should not be critical.
I could imagine that the difficulty is name space collision.
If the philosohpy is that the user shouldn’t know about the extension module or its name it’s not possible to import the types.
People that tend to only import modules in order to keep their name space clean would need to know the name of the extension module.
So the best solution might be to provide a macro in the parent module that brings the newly defined types
in the name space of the parent module.
Users would need to explicitly call that macro.
You want to export symbols that are conditionally defined based on what packages have been loaded in the session? Just checking so I understand.
You could do something like:
module Parent
export get_thingy
function get_thingy()
ext = Base.get_extension(@__MODULE__, :Extension)
if ext === nothing
error("extension not loaded...")
else
return ext.get_thingy()
end
end
Thanks for your answer, this is what I had in mind. But I would rather return a type than a function.
module Parent
export get_thingy, @get_thingy
function get_thingy()
ext = Base.get_extension(@__MODULE__, :Extension)
if ext === nothing
error("extension not loaded...")
else
return ext.MyExtensionType
end
end
macro get_thingy()
ext = Base.get_extension(@__MODULE__, :Extension)
MyType = if ext === nothing
error("extension not loaded...")
else
ext.MyExtensionType
end
:(const MyExtensionType = $MyType) |> esc
end
# Main
const MyExtensionType = get_thingy()
# alternatively (not tested yet, but along these lines)
@get_thingy
I could imagine official extension macros @import and @using with the following syntax
# import var1 and var2 from ParentExtension1Extension2Ext
@import Parent, Extension1, Extension2: var1, var2 as myvar 2
# import all exports from ParentExtension1Extension2Ext
@using Parent, Extension1, Extension2
# import all exports from all loaded Extensions
@using Parent __all
Does something along these lines sound reasonable to you?
You cannot have ext = Base.get_extension(@__MODULE__, :Extension) outside the macro expansion, it needs to be inside the returned expression.
Extensions are a run time thing, they might be loaded or they might not be so you have to check. Therefore, you cannot make a decision about them during precompile time (or macro expansion time). That’s why you need to run some code (like a function) to interact with them and you cannot really just export a type from it.
Maybe I was not very clear with my last post. I did not mean to use the macro from within the extension.
It could be called by the user after loading the parent and the extension module. This way the user could import variables or types from extensions without knowing the name of the extensions.
As an MWE I have prepared the following scenario:
packages Parent, Child1, Child2
extensions: ParentChild1Ext, ParentChild2Ext which export the types C1 and C2, respectively.
e.g.
module ParentChild1Ext
export C1
println("Child1 Extensions")
mutable struct C1
a::Int
b::Int
end
end
The following macro imports all exports of the extensions that have been loaded at that point of time
using TOML
macro import_extensions(parent, prefix = "")
prefix isa QuoteNode && (prefix = String(prefix.value))
toml = joinpath(dirname(dirname(pathof(@eval(__module__, $parent)))), "Project.toml")
extensions = get(TOML.parsefile(toml), "extensions", nothing)
ee = [Base.get_extension(Parent, Symbol(k)) for k in keys(extensions)]
ee = ee[ee .!== nothing]
output = quote end
for e in ee
nn = setdiff(names(e), [Symbol(e)])
for n in nn
n_new = Symbol(prefix, isempty(prefix) ? "" : "_", n)
if isdefined(__module__, n_new) && @eval(__module__, $n_new !== $e.$n)
@warn "$(repr(n)) already exists and could not be imported"
else
push!(output.args, :(const $n_new = $e.$n))
end
end
end
push!(output.args, :nothing)
:($output) |> esc
end
using Parent
using Child1
@import_extensions Parent
# imports C1
using Child2
@import_extensions Parent
# imports C2
@import_extensions Parent :hh
# imports C2 as hh_C2
f() = @import_extensions Parent
using Parent
using Child1
f()
and it doesn’t work. You are relying on the state of Julia at the time the macro is expanded. You can of course do that but that makes it tricky to use in a package for example, when the extension will be loaded after the macro expansion.
):