Well yes, but I do think that best practices should be to make the functionality exposed by overloading functions in the parent. It can even be a zero method function that is overloaded. So you would not need to access the namespace of the extension.
5 Likes
@kristoffer.carlsson, this is weird, I cannot get Base.get_extension to work in my REPL (but the tests are passing 
):
julia> using PGFPlotsX
julia> Base.get_extension(PGFPlotsX, :ColorsExt)
julia> using Colors
julia> Base.get_extension(PGFPlotsX, :ColorsExt) # what ?
julia> VERSION
v"1.9.0-beta3"
julia> "PGFPlotsX" ∈ map(x -> x.name, collect(keys(Base.loaded_modules)))
true
julia> "Colors" ∈ map(x -> x.name, collect(keys(Base.loaded_modules)))
true
julia> "ColorsExt" ∈ map(x -> x.name, collect(keys(Base.loaded_modules))) # what ?
false
What could have gone wrong here ?
(Occurred while working on this PR , CI is OK, but I cannot get it to work in the REPL locally) …
EDIT: hum, appears to be a duplicate of weakdeps don't load from fallback environment · Issue #48351 · JuliaLang/julia · GitHub, and should be resolved in next beta by allow extensions to be loaded from non top level env by KristofferC · Pull Request #48352 · JuliaLang/julia · GitHub.
I’d like to come back to question 1 of @cjdoris
I’m not sure whether this is answered in the above posts. I understand the question is about whether there is a possibility to export/import variables from the extension module.
In Reqires.jl it is possible to export variables. They are part of the parent module. For the extension module I have not been able to export or import variables, neither as part of the extension module, nor via @eval <parentmodule> export x, y as this would break precompilation.
Am I missing something?
1 Like
From what I understand, it is not considered to be a good practice to allow additional objects to be exported conditionally. For extensions, this kind of export behavior is not allowed at all.
1 Like
I’d like to learn more about this philosophy.
I think, it makes totally sense to have certain types available only if two (or more) packages are loaded.
I agree that they shouldn’t become part of the the parent module as it is the case with Require.jl, but leaving them in the extension module and allowing import/export should not be critical.
I could imagine that the difficulty is name space collision.
If the philosohpy is that the user shouldn’t know about the extension module or its name it’s not possible to import the types.
People that tend to only import modules in order to keep their name space clean would need to know the name of the extension module.
So the best solution might be to provide a macro in the parent module that brings the newly defined types
in the name space of the parent module.
Users would need to explicitly call that macro.
You want to export symbols that are conditionally defined based on what packages have been loaded in the session? Just checking so I understand.
You could do something like:
module Parent
export get_thingy
function get_thingy()
ext = Base.get_extension(@__MODULE__, :Extension)
if ext === nothing
error("extension not loaded...")
else
return ext.get_thingy()
end
end
4 Likes
Can it be
module Parent
export get_thingy
get_thingy() = error("Load package XXX")
end
module SomeExt
using XXX
import Parent.get_thingy
get_thingy() = actual code
end
?
The issue with that is that you are directly overwriting the method in Parent which is usually not a good idea.
2 Likes
Thanks for your answer, this is what I had in mind. But I would rather return a type than a function.
module Parent
export get_thingy, @get_thingy
function get_thingy()
ext = Base.get_extension(@__MODULE__, :Extension)
if ext === nothing
error("extension not loaded...")
else
return ext.MyExtensionType
end
end
macro get_thingy()
ext = Base.get_extension(@__MODULE__, :Extension)
MyType = if ext === nothing
error("extension not loaded...")
else
ext.MyExtensionType
end
:(const MyExtensionType = $MyType) |> esc
end
# Main
const MyExtensionType = get_thingy()
# alternatively (not tested yet, but along these lines)
@get_thingy
I could imagine official extension macros @import and @using with the following syntax
# import var1 and var2 from ParentExtension1Extension2Ext
@import Parent, Extension1, Extension2: var1, var2 as myvar 2
# import all exports from ParentExtension1Extension2Ext
@using Parent, Extension1, Extension2
# import all exports from all loaded Extensions
@using Parent __all
Does something along these lines sound reasonable to you?
You cannot have ext = Base.get_extension(@__MODULE__, :Extension) outside the macro expansion, it needs to be inside the returned expression.
Extensions are a run time thing, they might be loaded or they might not be so you have to check. Therefore, you cannot make a decision about them during precompile time (or macro expansion time). That’s why you need to run some code (like a function) to interact with them and you cannot really just export a type from it.
1 Like
Maybe I was not very clear with my last post. I did not mean to use the macro from within the extension.
It could be called by the user after loading the parent and the extension module. This way the user could import variables or types from extensions without knowing the name of the extensions.
As an MWE I have prepared the following scenario:
- packages
Parent,Child1,Child2 - extensions: ParentChild1Ext, ParentChild2Ext which export the types C1 and C2, respectively.
e.g.
module ParentChild1Ext
export C1
println("Child1 Extensions")
mutable struct C1
a::Int
b::Int
end
end
The following macro imports all exports of the extensions that have been loaded at that point of time
using TOML
macro import_extensions(parent, prefix = "")
prefix isa QuoteNode && (prefix = String(prefix.value))
toml = joinpath(dirname(dirname(pathof(@eval(__module__, $parent)))), "Project.toml")
extensions = get(TOML.parsefile(toml), "extensions", nothing)
ee = [Base.get_extension(Parent, Symbol(k)) for k in keys(extensions)]
ee = ee[ee .!== nothing]
output = quote end
for e in ee
nn = setdiff(names(e), [Symbol(e)])
for n in nn
n_new = Symbol(prefix, isempty(prefix) ? "" : "_", n)
if isdefined(__module__, n_new) && @eval(__module__, $n_new !== $e.$n)
@warn "$(repr(n)) already exists and could not be imported"
else
push!(output.args, :(const $n_new = $e.$n))
end
end
end
push!(output.args, :nothing)
:($output) |> esc
end
using Parent
using Child1
@import_extensions Parent
# imports C1
using Child2
@import_extensions Parent
# imports C2
@import_extensions Parent :hh
# imports C2 as hh_C2
Try
f() = @import_extensions Parent
using Parent
using Child1
f()
and it doesn’t work. You are relying on the state of Julia at the time the macro is expanded. You can of course do that but that makes it tricky to use in a package for example, when the extension will be loaded after the macro expansion.
1 Like
Thanks for your reply. I was experiencing exactly that when I tried to use it in a package.
But wouldn’t it be possible to create a new import statement that would do what the macro does at compile time?
1 Like
Sorry for necroposting but a similar question has just popped up for the 100th time, and the answer relied on a long lost Slack thread. I think it’s time to put this knowledge in the docs.
Anyone want to help with this PR or review it?
2 Likes
Cross-posting a related issue: KeyError on internal use of extensions · Issue #50028 · JuliaLang/julia · GitHub
The basic idea is that you could have a package in [deps] (so it would automatically be installed; without the user needing to do it themselves), but load it as an extension, such as by calling a function:
function call_zygote_function(arg)
Base.require(@__MODULE__, :Zygote)
Base.invokelatest(my_zygote_function, arg)
end
which should trigger a Zygote.jl-containing extension to load. However, this does not yet work due to how the extension mechanism requires the package to be in [weakdeps]. It sounds like there’s nothing fundamental preventing this from working though, so it could be a nice way to conditionally load code from within a package while taking advantage of precompilation.
I’ve been google circling around an answer to my question and it comes near this one. It seems to me there is a valid use case for new methods in an extension.
I want to have my module have only the relevant plotting routines for Plots.jl and Makie.jl and I don’t want them loaded with the package unless the user decides to do some plotting.
I’m having a lot of difficulty getting this to work cleanly. The Makie recipes need me to predefine an empty copy of the ultimate method for plotting the object in the parent module and then import it.
I haven’t quite figured it out for the RecipesBase plot overload.
Any thoughts on the best practice for this?
2 Likes
I agree here and I’ve circled too. Have you found the way to the force?
Not with the Recipes. I got the extensions to work in general though. I just didn’t use the recipes to do it. I just made up a function and called that.
1 Like