What's your workflow to develop packages with complex dependencies?

General Usage
, ,
1

Hi all,

I developed quite a few small packages with self-contained functionality, but inevitably during development I need complex dependencies (eg. Plots.jl) for functionality not required by the package itself, but required for e.g. illustrating usage, debugging, prototyping etc.

Unfortunately (and this is not a comment agains Plots) packages such as Plots have complex dependencies that (for me) can occassionally break in interesting, non-trivial ways. I also like to keep my source packages clean so the corresponding CI/CD/Container images are equally sparse.

Which brings me to my question, what is your workflow for this kind of scenario? Do you keep the complex dependencies in your base/main environment (but this can clash), or do you keep it all in the package with optional flags?
Or is there a middle way by stacking environments?

Any tips or suggestions are very welcome,

Ben

2

For me in the REPL it’s temp environments + autoinstall in 99% of cases.
Basically, do using SomePackage, and Julia suggests to install it for you. Press [o] and it’s installed into a temp env.
Unfortunately, not out of the box – requires adding a line to your startup.jl. See add "t" option for autoinstall into a temp env by aplavin · Pull Request #3867 · JuliaLang/Pkg.jl · GitHub.

Aside from REPL, and especially for plots, notebooks are very convenient. There, all extra deps are managed automatically: Pluto adds packages automatically when you do using SomePackage.

2 Likes
3

For one-off debugging et cetera, a temp environment that devs my package:

(@v1.10) pkg> activate --temp
  Activating new project at `/var/folders/57/1z4jgn5133lbbbrnnt4vf_zw0000gn/T/jl_IoTzyY`

(jl_IoTzyY) pkg> add Plots
   Resolving package versions...
[...]

(jl_IoTzyY) pkg> dev path/to/MyPackage

julia> # do stuff with Plots and MyPackage

For more persistent needs, the same, but rather than a temp environment, I create a regular env in a subfolder like path/to/MyPackage/scripts. If you make sure to use the relative path, dev .., this solution is portable and allows you to copy the project around, push to github, et cetera, without breaking the scripts environment.

Or, as @aplavin said, a Pluto.jl notebook somewhere like path/to/MyPackage/notebooks, where the first cell looks something like

begin
    import Pkg
    Pkg.dev(path="..")
    Pkg.add(["Plots"])
end
4 Likes
4

I would add it as test dependency:

[extras]
BenchmarkTools = "6e4b80f9-dd63-53aa-95a3-0cdb28fa8baf"
Plots = "23c2ee80-7a9e-4350-b264-8e670f12517c"
LaTeXStrings = "b964fa9f-0449-5b57-a5c2-d3ea65f4040f"
Test = "8dfed614-e22c-5e08-85e1-65c5234f0b40"

[targets]
test = ["Test", "Plots", "BenchmarkTools", "LaTeXStrings"]

If you have “TestEnv” in your global environment you can then do

using TestEnv; TestEnv.activate()

and Plots is available.

4 Likes
5

For exactly this use case I use shared / stacked environments, with ShareAdd.jl package (disclosure: I’m the author) to simplify their usage and management.

2 Likes
6

I think this is spot on for my needs @Eben60, will take a deeper look. @ufechner7 @danielwe @aplavin appreciate your suggestions as well!