### Author:  [@tfiers](https://discourse.julialang.org/u/tfiers)
#### Post date: [January 23, 2023, 12:14pm UTC](https://discourse.julialang.org/t/package-extensions-for-julia-1-9/93397/1 "2023-01-23T12:14:24Z")
Package extensions are an exciting new feature of julia 1.9
> **{links for more info about them}**
>
> - [julia/NEWS.md at v1.9.0-beta3 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/blob/v1.9.0-beta3/NEWS.md#package-manager)
> - [https://pkgdocs.julialang.org/dev/creating-packages/#Conditional-loading-of-code-in-packages-(Extensions)](https://pkgdocs.julialang.org/dev/creating-packages/#Conditional-loading-of-code-in-packages-(Extensions))
> - [Are extension packages importable? - #2 by mkitti](https://discourse.julialang.org/t/are-extension-packages-importable/92527/2) (links to implementation PRs)
They could e.g. be used to add Unitful support to Distributions.jl, without impacting package load-time for Distributions.jl users that do not need physical units-support. ([Unitful support? · Issue #1413 · JuliaStats/Distributions.jl · GitHub](https://github.com/JuliaStats/Distributions.jl/issues/1413#issuecomment-1400228821))
But I wondered how this works for people with a lower Julia version.
What happens if a Project.toml has these `[weakdeps]` and `[extensions]` sections, but your Julia version is 1.6, or 1.0?
(1.0 is still supported by Distributions.jl)
---
### Author:  [@albheim](https://discourse.julialang.org/u/albheim)
#### Post date: [January 23, 2023, 12:41pm UTC](https://discourse.julialang.org/t/package-extensions-for-julia-1-9/93397/2 "2023-01-23T12:41:57Z")
Seems to be a line about that in the [dev docs](https://pkgdocs.julialang.org/dev/creating-packages/#Using-an-extension-while-supporting-older-Julia-version) you linked.
> If you want to use an extension with compatibility constraints while supporting earlier Julia versions you have to duplicate the packages under `[weakdeps]` into `[extras]`. This is an unfortunate duplication but without doing this the project verifier under older Julia versions will complain (error).
---
### Author:  [@tfiers](https://discourse.julialang.org/u/tfiers)
#### Post date: [January 23, 2023, 12:45pm UTC](https://discourse.julialang.org/t/package-extensions-for-julia-1-9/93397/3 "2023-01-23T12:45:32Z")
Oh my bad, it’s literally right there lol
---
### Author:  [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)
#### Post date: [January 24, 2023, 3:02pm UTC](https://discourse.julialang.org/t/package-extensions-for-julia-1-9/93397/4 "2023-01-24T15:02:11Z")
> [@tfiers](#):
>
> They could e.g. be used to add Unitful support to Distributions.jl, without impacting package load-time for Distributions.jl users that do not need physical units-support. ([Unitful support? · Issue #1413 · JuliaStats/Distributions.jl · GitHub](https://github.com/JuliaStats/Distributions.jl/issues/1413#issuecomment-1400228821))
>
> But I wondered how this works for people with a lower Julia version.
> What happens if a Project.toml has these `[weakdeps]` and `[extensions]` sections, but your Julia version is 1.6, or 1.0?
> (1.0 is still supported by Distributions.jl)
I had a bit of trouble figuring it out the first time, but now I have an example of it completed:
>
It’s very formulaic to follow that and just move unitful support to extensions. What DiffEqBase is doing supports both Requires and extensions, and which one it uses is dependent on whether it’s Julia v1.9 or not.
So backwards compatible, formulaic, and only a few minutes? People should start slapping extension packages everywhere!
---
### Author:  [@mbauman](https://discourse.julialang.org/u/mbauman)
#### Post date: [January 24, 2023, 3:43pm UTC](https://discourse.julialang.org/t/package-extensions-for-julia-1-9/93397/5 "2023-01-24T15:43:59Z")
Wow, that’s a really helpful guide — the merge commit diff is the most readable there: [Merge pull request #856 from SciML/weakdeps · SciML/DiffEqBase.jl@c2de920 · GitHub](https://github.com/SciML/DiffEqBase.jl/commit/c2de920a88a39814ac368f703c40fbc282a9c078)
My quick takeaways:
- Use both Requires (pre-1.9) and Extensions, using `isdefined(Base, :get_extension)` to determine which system to use
- Put the conditional dependencies in `[weakdeps]` and `[extras]` and `[test]` sections
- Extensions go in `ext/ExtensionExt.jl`, with a structure like:
```julia
module UnitfulExt
import DiffEqBase
isdefined(Base, :get_extension) ? (import Unitful) : (import ..Unitful)
# ... extensions here
end
```
(note that Requires plops the dependencies into the parent module, so you use `import ..Dependency`, whereas the builtin extensions support just makes it happen)
- Requires support for older versions has this ` __init__ ` for the parent package:
```julia
function __init__ ()
@static if !isdefined(Base, :get_extension)
@require Unitful="1986cc42-f94f-5a68-af5c-568840ba703d" begin
include("../ext/UnitfulExt.jl")
end
end
end
```
---
### Author:  [@jishnub](https://discourse.julialang.org/u/jishnub)
#### Post date: [January 24, 2023, 3:49pm UTC](https://discourse.julialang.org/t/package-extensions-for-julia-1-9/93397/6 "2023-01-24T15:49:47Z")
> [@albheim](#):
>
> you have to duplicate the packages under `[weakdeps]` into `[extras]`.
But the commit above doesn’t seem to add to `[extras]`?
---
### Author:  [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)
#### Post date: [January 24, 2023, 3:52pm UTC](https://discourse.julialang.org/t/package-extensions-for-julia-1-9/93397/7 "2023-01-24T15:52:58Z")
> [@jishnub](#):
>
> But the commit above doesn’t seem to add to `[extras]`?
It does. It’s a little quirk that one has to make a note of. There is a note in the docs about it:
[https://pkgdocs.julialang.org/dev/creating-packages/#Using-an-extension-while-supporting-older-Julia-versions](https://pkgdocs.julialang.org/dev/creating-packages/#Using-an-extension-while-supporting-older-Julia-versions)
That I missed probably the first 3 times trying to figure out what was wrong. Backwards compatibility tests will fail until that’s done.
---
### Author:  [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)
#### Post date: [January 24, 2023, 4:03pm UTC](https://discourse.julialang.org/t/package-extensions-for-julia-1-9/93397/8 "2023-01-24T16:03:05Z")
Might be good to distill this into a “worked example” of using extensions while supporting older Julia versions. Also wonder if any of this could be streamlined a bit now that it works
---
### Author:  [@mbauman](https://discourse.julialang.org/u/mbauman)
#### Post date: [January 24, 2023, 4:10pm UTC](https://discourse.julialang.org/t/package-extensions-for-julia-1-9/93397/9 "2023-01-24T16:10:45Z")
The diff rendering oh-so-helpfully collapses the `[extras]` section header and doesn’t extract the header correctly in its preview:
---
### Author:  [@cjdoris](https://discourse.julialang.org/u/cjdoris)
#### Post date: [January 24, 2023, 6:09pm UTC](https://discourse.julialang.org/t/package-extensions-for-julia-1-9/93397/10 "2023-01-24T18:09:58Z")
> [@ChrisRackauckas](#):
>
> So backwards compatible, formulaic, and only a few minutes? People should start slapping extension packages everywhere!
In fact, it’s so formulaic that you can turn it into a macro for set-and-forget backwards compatibility. Here’s a demo package I just whipped up:
> **[GitHub - cjdoris/PackageExtensionCompat.jl: Makes Julia's package extensions...](https://github.com/cjdoris/PackageExtensionCompat.jl)**
>
> Makes Julia's package extensions backwards compatible - GitHub - cjdoris/PackageExtensionCompat.jl: Makes Julia's package extensions backwards compatible
Edit: Maybe this macro should be added to Compat.jl instead?
---
### Author:  [@ToucheSir](https://discourse.julialang.org/u/ToucheSir)
#### Post date: [January 24, 2023, 8:47pm UTC](https://discourse.julialang.org/t/package-extensions-for-julia-1-9/93397/11 "2023-01-24T20:47:15Z")
This is more a feature request than a docs one, but one case which hasn’t been covered is how to migrate an existing glue package into an extension. Some of these packages have their own dependencies separate from the weak deps, so those are probably out. Others are large enough that converting them into extensions with a Requires.jl fallback for \<1.9 isn’t feasible because one misses out on precompilation. The only complete suggestions I’ve seen so far for this are either to fork one’s codebase into pre-1.9 and post-1.9 versions, duplicate code between extension and glue package, or to wait for a Julia version with package extensions to become LTS.
---
### Author:  [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)
#### Post date: [January 24, 2023, 9:54pm UTC](https://discourse.julialang.org/t/package-extensions-for-julia-1-9/93397/12 "2023-01-24T21:54:05Z")
I think we may have covered this in a Slack discussion but haven’t put it online yet? The answer is that it’s a missing feature right now. The posted way in Slack to do this was to make the glue package be a dependency of the original package and then make the extension module reexport using the glue package. That sounded great, but when I went to try and go do that, I realized it did not work because it introduced a circular dependency, and such circular dependencies are not allowed in Pkg.
To bring this down to something more concrete, DiffEqBase is the core of differential equations, Zygote does automatic differentiation, and SciMLSensitivity adds adjoint overloads to the differential equation solvers for automatic differentiation. Right now if you try to use Zygote on a differential equation without `using SciMLSensitivity`, you get an error telling you to `using SciMLSensitivity`, so the improvement would be to use the package extension functionality to automatically load SciMLSensitivity when Zygote and DiffEqBase are both loaded. The suggested way to do this was to make SciMLSensitivity into a dependency of DiffEqBase and then make a ZygoteExt whose only code is `@reexport using SciMLSensitivity`. This would make SciMLSensitivity get installed even when it isn’t needed, but it would only make it get `using`’d as required so it’s a partial solution.
But what I’m saying the issue is, is that if you try and make SciMLSensitivity into a dependency of DiffEqBase you get an error about having a circular dependency because (as a glue package) it has a dependency on DiffEqBase. @ToucheSir I know there’s the similar case of NNLibCUDA: there’s NNLib.jl and CUDA.jl and when both are in the environment you’d want to pull in NNLibCUDA.jl. But NNLibCUDA.jl will have a dependency on NNLib, so that will fail. If the circular dependency doesn’t exist then that would be a solution though.
But that’s at least my current understanding of what @kristoffer.carlsson was suggesting, maybe I wasn’t understanding it correctly or he has a nice way to get around that limitation.
---
### Author:  [@fredrikekre](https://discourse.julialang.org/u/fredrikekre)
#### Post date: [January 24, 2023, 10:04pm UTC](https://discourse.julialang.org/t/package-extensions-for-julia-1-9/93397/13 "2023-01-24T22:04:26Z")
> [@mbauman](#):
>
> Extensions go in `ext/ExtensionExt.jl`, with a structure like
Note that it doesn’t have to be named like like that. Although the extension module is in most cases invisible to users, it might show up in e.g. stacktraces. I think a more descriptive name might be useful, for example `DiffEqBaseUnitful` for `DiffEqBase`s extension to `Unitful`. (I can imagine many package having `StaticArraysExt` for example, and perhaps it might be confusing in some cases.)
> [@ToucheSir](#):
>
> Some of these packages have their own dependencies separate from the weak deps, so those are probably out.
Note that an extension can be conditional on more than just one package.
---
### Author:  [@ToucheSir](https://discourse.julialang.org/u/ToucheSir)
#### Post date: [January 24, 2023, 10:08pm UTC](https://discourse.julialang.org/t/package-extensions-for-julia-1-9/93397/14 "2023-01-24T22:08:25Z")
@ChrisRackauckas We covered parts but not all of this across a few different Slack threads (I think you weren’t on all of them), so I figured it would be best to put it in one public place to get more thoughts.
The extension package has deps issue is actually the least worrisome of the two for me. e.g. for the NNlibCUDA example, that’s already a hard dep of Flux so we could easily follow the hard dep → weak dep migration path outlined in the docs (@fredrikekre we want to explicitly [avoid being conditional](https://github.com/FluxML/Flux.jl/pull/2132#issuecomment-1345643671) on the glue package here because it’s, well, a glue package).
The bigger challenge I see is the second one, which is what happens when you have a library with multiple “backends” represented as extensions that _weren’t_ previously hard dependencies. I tried this in [Re-integrate NNlibCUDA as a package extension by ToucheSir · Pull Request #445 · FluxML/NNlib.jl · GitHub](https://github.com/FluxML/NNlib.jl/pull/445), but it turns out the precompilation hit from Requires is so large that it negates any benefit of using extensions pre 1.9. After a long back and forth on Slack, it didn’t seem like anyone had a reliable “polyfill” for this that enabled precompilation on \<1.9, and so I stopped looking into it altogether.
---
### Author:  [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)
#### Post date: [January 25, 2023, 2:16am UTC](https://discourse.julialang.org/t/package-extensions-for-julia-1-9/93397/15 "2023-01-25T02:16:19Z")
> [@fredrikekre](#):
>
> Note that it doesn’t have to be named like like that. Although the extension module is in most cases invisible to users, it might show up in e.g. stacktraces. I think a more descriptive name might be useful, for example `DiffEqBaseUnitful` for `DiffEqBase`s extension to `Unitful`. (I can imagine many package having `StaticArraysExt` for example, and perhaps it might be confusing in some cases.)
Good point, and as a point of style we should probably stick to a standard rule. I think XY where X is the package that is being extended and Y is the trigger/glue is a pretty good rule, like what you show with DiffEqBaseUnitful. We should probably rename our extensions to match that.
---
### Author:  [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)
#### Post date: [January 25, 2023, 2:17am UTC](https://discourse.julialang.org/t/package-extensions-for-julia-1-9/93397/16 "2023-01-25T02:17:46Z")
> [@ToucheSir](#):
>
> The bigger challenge I see is the second one, which is what happens when you have a library with multiple “backends” represented as extensions that _weren’t_ previously hard dependencies. I tried this in [Re-integrate NNlibCUDA as a package extension by ToucheSir · Pull Request #445 · FluxML/NNlib.jl · GitHub](https://github.com/FluxML/NNlib.jl/pull/445), but it turns out the precompilation hit from Requires is so large that it negates any benefit of using extensions pre 1.9. After a long back and forth on Slack, it didn’t seem like anyone had a reliable “polyfill” for this that enabled precompilation on \<1.9, and so I stopped looking into it altogether.
Make sure this all gets captured in issues?
---
### Author:  [@ToucheSir](https://discourse.julialang.org/u/ToucheSir)
#### Post date: [January 25, 2023, 3:06am UTC](https://discourse.julialang.org/t/package-extensions-for-julia-1-9/93397/17 "2023-01-25T03:06:24Z")
Where exactly? The description in that PR is pretty self-contained, and links to [CategoricalArrays forces OpenSpecFun\_jll recompilation - #18 by tim.holy](https://discourse.julialang.org/t/categoricalarrays-forces-openspecfun-jll-recompilation/87759/18) and [Warning when conditionally loading "glue" module · Issue #65 · JuliaPackaging/Requires.jl · GitHub](https://github.com/JuliaPackaging/Requires.jl/issues/65). Which themselves transitively link to [Warning when conditionally loading "glue" module · Issue #1238 · JuliaLang/Pkg.jl · GitHub](https://github.com/JuliaLang/Pkg.jl/issues/1238) (closed as for Base), [Warning when conditionally loading "glue" module · Issue #32413 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/issues/32413) (no follow-up), [Thoughts on how we can make Requires precompile friendly. · Issue #33 · JuliaPackaging/Requires.jl · GitHub](https://github.com/JuliaPackaging/Requires.jl/issues/33) (closed in favour of Discourse topic) and [Optional dependencies / Requires.jl](https://discourse.julialang.org/t/optional-dependencies-requires-jl/3294) (contains relevant discussion but out of date). Oh, and a Slack thread where I tested the ideas in Requires.jl#65 but didn’t get much engagement after mentioning they failed.
It would be ideal if we could avoid forking this discussion onto another thread and instead continue here/consolidate it into an existing issue.
---
### Author:  [@tim.holy](https://discourse.julialang.org/u/tim.holy)
#### Post date: [January 25, 2023, 10:19am UTC](https://discourse.julialang.org/t/package-extensions-for-julia-1-9/93397/18 "2023-01-25T10:19:46Z")
Personally I would say “meh” about working too hard for Julia versions prior to 1.9. Packages should function _correctly_ on all Julia versions, but if a recent Julia version provides the best _experience_ I don’t think we need to kill ourselves trying to match it on older versions, since we can just tell people to use the latest release. (Yes, I know 1.9 is not yet released…)
If you want to make changes that optimize the 1.9+ experience but significantly degrade the experience on older Julia versions, to me that sounds like time for a new minor (or breaking, if necessary) release and minimum bounds on the Julia version. In weird cases you might need [RetroCap](https://github.com/JuliaRegistries/RetroCap.jl) or something. That will allow you to keep users on the older versions until they adopt a Julia version optimized for the new design.
In my view we should play the long game, rather than going to a lot of effort to make improvements that will have a transient lifetime.
---
### Author:  [@ToucheSir](https://discourse.julialang.org/u/ToucheSir)
#### Post date: [January 25, 2023, 3:13pm UTC](https://discourse.julialang.org/t/package-extensions-for-julia-1-9/93397/19 "2023-01-25T15:13:43Z")
Thanks Tim. Given we still receive issues from users on 1.7.x (including from orgs using Julia. I presume upgrading production systems and/or clients takes time) and Flux’s previous less-than-stellar experience using backport branches, it looks like this is going to be a longer and more incremental transition than we originally expected. If anyone is interested, I can try to keep this topic updated as we make progress.
---
### Author:  [@t-bltg](https://discourse.julialang.org/u/t-bltg)
#### Post date: [February 5, 2023, 9:46am UTC](https://discourse.julialang.org/t/package-extensions-for-julia-1-9/93397/20 "2023-02-05T09:46:07Z")
> [@mbauman](#):
>
> ```julia
> module UnitfulExt
>
> import DiffEqBase
> isdefined(Base, :get_extension) ? (import Unitful) : (import ..Unitful)
> # ... extensions here
> end
>
> ```
I find the conditional on imports cumbersome (duplicate list). How can we avoid these kind of duplicates:
```julia
@static if isdefined(Base, :get_extension)
import Unitful: Quantity, RealOrRealQuantity, ustrip, unit
else
import ..Unitful: Quantity, RealOrRealQuantity, ustrip, unit
end
```
EDIT: 🤣
```julia
macro rel(imp_use::QuoteNode, mod::Symbol, args...)
dots = ntuple(_ -> :., isdefined(Base, :get_extension) ? 1 : 3)
ex = if length(args) > 0
Expr(:(:), Expr(dots..., mod), Expr.(:., args)...)
else
Expr(dots..., mod)
end
Expr(imp_use.value, ex) |> esc
end
println(@macroexpand @rel(:import, Unitful, Quantity, RealOrRealQuantity))
@rel :import Unitful Quantity RealOrRealQuantity
println(@macroexpand @rel(:using, Unitful))
@rel :using Unitful
```
[Next page](https://discourse.julialang.org/t/package-extensions-for-julia-1-9/93397.md?page=2)