Question about new "package extensions"?

General Usage
1

I realize some of the docs may still be getting filled in prior to 1.9 release, but couldn’t resist asking about this potentially really nice new feature. Two questions:

  1. Say my package Foo has an optional dependency on CUDA.jl (as I understand specified by a [weakdeps] entry). When I add Foo, does CUDA get installed?

  2. Say Foo has a weakdep on CUDA and a corresponding compat entry CUDA = "4" but I try to install Foo into a parent environment which is explicitly restricted to CUDA = "3". Can Foo still be installed here? Just the package extension will not be loadable?

2

No. The point is that the user can opt into the extended functionality by adding CUDA. The “entrypoint” to the extension for this case would probably be to pass a CUDA array, so then you have, of course, installed CUDA.

Yes, and yes, since the weak dependencies are not hard requirements.

6 Likes
3

Would there be a way for the user to know he needs to update Foo in the following situation? CUDA@4 and Foo@1.1 are installed, Foo@1.1 has a weakdep on CUDA@3 and Foo@1.2 weakdeps on CUDA@4.
I can imagine lots of confusion in such cases.

4

Awesome, that’s exactly the answers I was hoping to get!

I do agree with @aplavin, it would a nice addition if there was a way for the user to know that a compat bound is preventing a package extension from loading. Maybe exposed in pkg> st -m?

Two other minor things that came up while I was trying this out briefly:

  1. For a package I’m developing Foo, if I using Foo and it errors, in the same session I can fix the error and try again until it loads. For package extensions, if it errors, is there a way to trigger trying to reload it? Or do I have to restart my session? Would be nice not to have to.

  2. With @require one thing I found nice was the ability to trigger exports in the main package. Like:

    module Foo
@require Bar="..." begin
    some_foo_bar_specific_function(x) = ...
    export some_foo_bar_specific_function
end

    Is there anything like this with package extensions?

If I file any of these as issues, is the right place the julia or Pkg.jl repos? In any case, really great to have package extensions in the main language, super useful!

5

There is status --extensions which will show some extension related information, so you will at least see if the extension is active or not.

Base.retry_load_extensions()

Not really. The main “entrypoint” of an extension is new method dispatches. You can define and empty function and export it regardless though. Me and @kristoffer.carlsson have discussed this quite a bit so maybe he will chime in. Some of this is also discussed in JuliaLang/Pkg.jl#3282.

Pkg.jl probably, to start with.

2 Likes
6

Just double checked this and at least on 1.9-rc1 the answer here is actually no. Wondering if I did something wrong or this is intentional?

A MWE, I have a package Foo with Project.toml:

name = "Foo"

[weakdeps]
CUDA = "052768ef-5323-5732-b1bb-66c8b64840ba"

[compat]
CUDA = "4"

Now from another environment I dev Foo, so that status shows:

(test) pkg> st
Status `~/Project.toml`
  [37d5fa7c] Foo v0.1.0 `Foo`

But if I try to add CUDA@3 I get:

(test) pkg> add CUDA@3
   Resolving package versions...
ERROR: Unsatisfiable requirements detected for package CUDA [052768ef]:
 CUDA [052768ef] log:
 ├─possible versions are: 0.1.0-4.1.2 or uninstalled
 ├─restricted to versions 4 by Foo [37d5fa7c], leaving only versions: 4.0.0-4.1.2 or uninstalled
 │ └─Foo [37d5fa7c] log:
 │   ├─possible versions are: 0.1.0 or uninstalled
 │   └─Foo [37d5fa7c] is fixed to version 0.1.0
 └─restricted to versions 3 by an explicit requirement — no versions left

So it seems like here the weakdep is a hard requirement?

2 Likes
7

To relay some discussion from Slack here for posterity, this is indeed the intended behavior, compat entries for weakdeps are hard requirements.

If in this example if you want to allow having Foo and CUDA@3 in the same environment, one workaround is to relax the compat bound to CUDA = "3" but in the extension module do a version-check and if CUDA < v"4" then don’t actually load any of the extension code.

3 Likes