Are extension packages importable?

The new extension packages functionality looks great! Code Loading · The Julia Language

I get that its intended use is to provide Requires.jl-like functionality, i.e. to run some code when some other package is loaded, e.g. in order to add compatibility methods.

Two questions:

  1. Is there a mechanism to import the module defined in an extension package?
  2. Is there a mechanism to install the packages for an extension? E.g. if package Foo had an extension FooExt depending on ExtPkg say, can I specify to install FooExt somehow (which implicitly installs ExtPkg) instead of specifying ExtPkg explicitly?

If not, are these on the roadmap?

The second suggestion is a lot like Python’s “extras”, which you’d install like

pip install 'Foo[FooExt]'

My vague understanding is that extension packages are true packages on their own. You could probably add them as subdirectory packages.

I think @kristoffer.carlsson would be the one most qualified to clarify this.

Here is the pull request to Julia:

and the corresponding pull request to Pkg.jl:

Here are some example extension modules:

It appears that these modules do not have UUIDs so they may not be bonafide packages in themselves. The example prototypes show the extensions are usually modules which are included as submodules of the base package.

1 Like

You can get the module using get_extension. Using PGFPlotsX as an example:

julia> using PGFPlotsX

julia> Base.get_extension(PGFPlotsX, :ColorsExt)

julia> using Colors

julia> Base.get_extension(PGFPlotsX, :ColorsExt)

So before the extension is loaded get_extension will return nothing.

In general though, you should probably try to make getting the module not required to use the package. It is better to try have the extension add methods via dispatch and make the new functionality available in that way.

They get their UUID by combining the UUID of the parent package and the name of the extension module:

# Get UUID of extension module
julia> Base.PkgId(Base.get_extension(PGFPlotsX, :ColorsExt)).uuid

# Which is computed like this:
julia> Base.uuid5(Base.identify_package("PGFPlotsX").uuid, "ColorsExt")

Thank you for the replies!

I get your point for the intended usage of extensions, I was more wondering if the same mechanism could be co-opted to provide something a bit like Python’s “extras”.

In a way I’m asking if it’s possible to have multiple modules in a single Julia package which can be independently imported and have separate dependencies. The answer to that is seemingly currently “no” - which is fine because you can always create multiple packages instead.

To be concrete, suppose the package Foo has an extension Ext which depends on Bar and Baz.

Then it may be nice to let

pkg> add Foo[Ext]

be a shorthand for

pkg> add Foo, Bar, Baz

(thus allowing the fact that Ext depends on Bar and Baz to be an implementation detail).

And further, it may be nice to let

import Foo[Ext]

be a shorthand for

import Foo, Bar, Baz
const Ext = Base.get_extension(Foo, :Ext)

It still isn’t clear to me if the name of the extension should even be something a user should ever know about.


I guess like everything else, it’s up to the package author whether or not a particular extension is part of the API.

Well yes, but I do think that best practices should be to make the functionality exposed by overloading functions in the parent. It can even be a zero method function that is overloaded. So you would not need to access the namespace of the extension.


@kristoffer.carlsson, this is weird, I cannot get Base.get_extension to work in my REPL (but the tests are passing :confused: ):

julia> using PGFPlotsX

julia> Base.get_extension(PGFPlotsX, :ColorsExt)

julia> using Colors

julia> Base.get_extension(PGFPlotsX, :ColorsExt)  # what ?

julia> VERSION

julia> "PGFPlotsX" ∈ map(x ->, collect(keys(Base.loaded_modules)))

julia> "Colors" ∈ map(x ->, collect(keys(Base.loaded_modules)))

julia> "ColorsExt" ∈ map(x ->, collect(keys(Base.loaded_modules)))  # what ?

What could have gone wrong here ?

(Occurred while working on this PR , CI is OK, but I cannot get it to work in the REPL locally) …

EDIT: hum, appears to be a duplicate of weakdeps don't load from fallback environment · Issue #48351 · JuliaLang/julia · GitHub, and should be resolved in next beta by allow extensions to be loaded from non top level env by KristofferC · Pull Request #48352 · JuliaLang/julia · GitHub.

I’d like to come back to question 1 of @cjdoris

I’m not sure whether this is answered in the above posts. I understand the question is about whether there is a possibility to export/import variables from the extension module.

In Reqires.jl it is possible to export variables. They are part of the parent module. For the extension module I have not been able to export or import variables, neither as part of the extension module, nor via @eval <parentmodule> export x, y as this would break precompilation.

Am I missing something?

1 Like

same question as @hhaensel

From what I understand, it is not considered to be a good practice to allow additional objects to be exported conditionally. For extensions, this kind of export behavior is not allowed at all.

1 Like

I’d like to learn more about this philosophy.
I think, it makes totally sense to have certain types available only if two (or more) packages are loaded.
I agree that they shouldn’t become part of the the parent module as it is the case with Require.jl, but leaving them in the extension module and allowing import/export should not be critical.

I could imagine that the difficulty is name space collision.

If the philosohpy is that the user shouldn’t know about the extension module or its name it’s not possible to import the types.
People that tend to only import modules in order to keep their name space clean would need to know the name of the extension module.

So the best solution might be to provide a macro in the parent module that brings the newly defined types
in the name space of the parent module.
Users would need to explicitly call that macro.

You want to export symbols that are conditionally defined based on what packages have been loaded in the session? Just checking so I understand.

You could do something like:

module Parent

export get_thingy

function get_thingy()
    ext = Base.get_extension(@__MODULE__, :Extension)
    if ext === nothing
        error("extension not loaded...")
        return ext.get_thingy()

Can it be

module Parent
export get_thingy
get_thingy() = error("Load package XXX")

module SomeExt
using XXX
import Parent.get_thingy
get_thingy() = actual code


The issue with that is that you are directly overwriting the method in Parent which is usually not a good idea.


Thanks for your answer, this is what I had in mind. But I would rather return a type than a function.

module Parent

export get_thingy, @get_thingy

function get_thingy()
    ext = Base.get_extension(@__MODULE__, :Extension)
    if ext === nothing
        error("extension not loaded...")
        return ext.MyExtensionType

macro get_thingy()
    ext = Base.get_extension(@__MODULE__, :Extension)
    MyType = if ext === nothing
        error("extension not loaded...")
    :(const MyExtensionType = $MyType) |> esc

# Main

const MyExtensionType = get_thingy()

# alternatively (not tested yet, but along these lines)

I could imagine official extension macros @import and @using with the following syntax

# import var1 and var2 from ParentExtension1Extension2Ext
@import Parent, Extension1, Extension2: var1, var2 as myvar 2

# import all exports from ParentExtension1Extension2Ext
@using Parent, Extension1, Extension2

# import all exports from all loaded Extensions
@using Parent __all

Does something along these lines sound reasonable to you?

You cannot have ext = Base.get_extension(@__MODULE__, :Extension) outside the macro expansion, it needs to be inside the returned expression.

Extensions are a run time thing, they might be loaded or they might not be so you have to check. Therefore, you cannot make a decision about them during precompile time (or macro expansion time). That’s why you need to run some code (like a function) to interact with them and you cannot really just export a type from it.

1 Like

Maybe I was not very clear with my last post. I did not mean to use the macro from within the extension.
It could be called by the user after loading the parent and the extension module. This way the user could import variables or types from extensions without knowing the name of the extensions.

As an MWE I have prepared the following scenario:

  • packages Parent, Child1, Child2
  • extensions: ParentChild1Ext, ParentChild2Ext which export the types C1 and C2, respectively.


module ParentChild1Ext

export C1
println("Child1 Extensions")

mutable struct C1


The following macro imports all exports of the extensions that have been loaded at that point of time

using TOML

macro import_extensions(parent, prefix = "")
    prefix isa QuoteNode && (prefix = String(prefix.value))
    toml = joinpath(dirname(dirname(pathof(@eval(__module__, $parent)))), "Project.toml")
    extensions = get(TOML.parsefile(toml), "extensions", nothing)
    ee = [Base.get_extension(Parent, Symbol(k)) for k in keys(extensions)]
    ee = ee[ee .!== nothing]
    output = quote end
    for e in ee
        nn = setdiff(names(e), [Symbol(e)])
        for n in nn
            n_new = Symbol(prefix, isempty(prefix) ? "" : "_", n)
            if isdefined(__module__, n_new) && @eval(__module__, $n_new !== $e.$n)
                @warn "$(repr(n)) already exists and could not be imported"
                push!(output.args, :(const $n_new = $e.$n))
    push!(output.args, :nothing)
    :($output) |> esc

using Parent
using Child1

@import_extensions Parent
# imports C1

using Child2

@import_extensions Parent
# imports C2

@import_extensions Parent :hh
# imports C2 as hh_C2


f() = @import_extensions Parent

using Parent
using Child1


and it doesn’t work. You are relying on the state of Julia at the time the macro is expanded. You can of course do that but that makes it tricky to use in a package for example, when the extension will be loaded after the macro expansion.

1 Like