[ANN] ShareAdd.jl - making easy to import packages from multiple environments

Package Announcements
, , , ,
1

I am happy to announce the newly registered package ShareAdd.jl .

The package is intended for the interactive use. It’s purpose is to make commonly used packages comfortably available without the dilemma of either adding them to the package under development or keeping them in the “main” Julia environment. Instead, these packages can be kept in shared environments, either grouped, or one package per environment, which would be added to the LOAD_PATH as necessary.

ShareAdd.jl exports macro @usingany. This macro makes package(s) available, if they are not already, and loads them with using keyword:

  • If a package is available in any environment in LOAD_PATH, that’s OK.
  • If a package is available in any of the shared environments, this environment will be pushed into LOAD_PATH.
  • Otherwise if it can be installed, you will be prompted to select or create an environment to install the package(s).
  • If the package is not listed in any registry, an error will be thrown.

The package also exports several utility functions.

For further details, see documentation.

15 Likes
2

Version 1.0 is now registered.

  • Improved using syntax variations support - this is now OK, too:
@usingany Foo: bar
  • Added the macro @usingtmp for those who prefer to use temporary environments:
@usingtmp Foo

will activate a temporary environment, dev there the package that was under development (if there was any) before the macro call, install Foo into this temporary env, and import it by using Foo.

  • Documentation made even better. :slight_smile:
  • Some minor usage improvement.
4 Likes
3

ShareAdd.jl can be combined nicely with BasicAutoloads.jl by @Lilith .

Here is my current startup.jl:


if VERSION >= v"1.11" && isinteractive()
	using ShareAdd
    import BasicAutoloads
    BasicAutoloads.register_autoloads([
    	["plot", "scatter"] => :(isdefined(Main, :plot) || @usingany Plots),
    	["DataFrame", "combine", "transform!", "transform", "select!", "select", "groupby"] 
    		=> :(@usingany DataFrames),
    	["CSV"] => :(@usingany CSV, StringEncodings, DataFrames),
    	["Dates", "Date","DateTime"] => :(@usingany Dates),
    	["mean", "std", "median"] => :(isdefined(Main, :mean) || @usingany Statistics), 
    	["@u_str"] => :(@usingany Unitful),  
    	["@benchmark"] => :(@usingany BenchmarkTools),   	
    ])
    println("ShareAdd & BasicAutoloads are loaded")
end

Now, should I e.g. type mean([1,2,3]) or 1.55u"V" in the REPL, Statistics.jl or resp. Unitful.jl would be first silently loaded.

The “main” shared environment, i.e. the folder “~/.julia/environments/v1.11/” contains now only Revise, ShareAdd, and BasicAutoloads. Most “shared” packages are each individually in the eponymous env, like Plots package is in @Plots shared env.

8 Likes
4

Exciting! This reminds me of a hack I cobbled together here, I’m glad to see it in a package that is more usable :slight_smile:

1 Like
5

This approach is something that I’ve experimented with in my config (I’ve actually got an autoloads setup that looks through shared environments automatically), but I ran into the issue that shared environments are version-specific.

The solution to this I’ve currently got is to have the sets of available packages defined in a TOML file, and have per-version shared environments generated from that spec in a separate folder that’s added to the load path.

6

Could you probably explain in more details what was going on?

Yes, sometimes a shared package gets recompiled, which is annoying with large packages like Plots.jl.

Now I’ve tried to look at it a bit systematically. I selected 8 packages I was working with in the last time, updated all their environments. Updated the shared env @Plots. Then

  • start Julia with the environment of the next package
  • @usingany Plots - which sometimes resulted in recompiling Plots
  • exit Julia

After I did it once with each package, I repeated the cycle once again. Now, there was no recompiles on this second use of @usingany Plots.

So, the good news is, a recompile with one package doesn’t influence others. Once you always have the same combination of “main package environment” and the set of “additional packages”, and made no updates, no recompiles are expected.

However if you got it recompiled with @usingany Plots., and another day do @usingany Makie, and still another day @usingany Plots - each time you change the plotting package, you may expect it being recompiled. Presumably one of them cause downgrading of some dependencies, which another one then upgrades again the next day. I’m not sure what can be done about it.

7

The real problem is that the resolved manifest with one Julia version will straight up be different to that of another Julia version: making global shared environments often incompatible with other Julia versions (how different many Julia versions work depending on the particular packages in the environment ).

8

Well, starting with Julia v1.11 it is possible to have separate manifests for different Julia versions. As v1.10 is now LTS, having non-version-specific manifest for v1.10 and version-specific ones for newer versions would probably cover the vast majority of use cases.

I’d implement creating version-specific manifests when using ShareAdd to update packages or environments. Thank you for drawing my attention to the issue :slightly_smiling_face:

P.S. Actually as ShareAdd.jl specifies Juvila version >= v1.10, it would cover 100% of cases supported by ShareAdd.jl