This package automtically create the environment and installs the needed packages when you run the Julia program (assuming “using QuickEnv” is at the top). Julia also has a neat feature of named environments, and QuickEnv search & find such an environment that the program can use to run in, and use it, instead of creating a new environment. So, for example, you can setup a named environment that is sufficient for most of your programs, and QuickEnv would automatically default to using this environment. QuickEnv also has a mechanism for setting up such named environments.
Since the default behavior of QuickEnv is to create a local environment in the current directory, this avoids the natural behavior of over-installing packages in the global environments. For more details, see here:
I think this is useful, but I would not be shocked if somebody came up with a more elegant solution. I just didn’t find it… (My previous solution was to have a different shabang that defaulted to running the program in the current directory environment.)
Local Directory Projects (--project=.): Reproducible, but creates directory clutter and repetitive setup overhead for single-file scripts or quick calculations. Furthermore, it results in additional disk space usage and compile-time overhead due to redundant package downloads and precompilations.
I don’t believe this is true.
For the “clutter” it is just a matter of:
using Pkg
PKG.activate(@__DIR__)
For the disk space, also I don’t believe it’s true: packages are not download d multiple times and precompiled package can be reused between environments (at least I think).
I’d say my package is a bit more verbose but also more principled, in that it includes the project file into the script, so you can have greater control over the project (like compat bounds and sources).
It’s not registered.
Here’s the example script from the readme:
# /// project
# name = "example"
#
# [deps]
# Example = "7876af07-990d-54b4-ab0e-23690620f79a"
# ///
using SelfContainedScripts
SelfContainedScripts.activate()
using Example
@show Example.hello("Alice")
I don’t necessarily disagree, but let me point out that “using QuickEnv” is easier. And as pointed out by the package description, since it uses a named environment if it can find one, it does save (maybe a bit) of disk space. It definitely saves runtime in such a case, as the script/program starts immediately if such a named environment exists.
As for the comments as directives. I think this is common. If there is a better alternative, I would be happy to use it.
It does feel a bit too “magic” for my tastes to be honest. I’d rather my package management be very explicit, and once you start going with those special comments then why not just have a full UV like approach instead? So being explicit rather than defining what you want in a roundabout way by tweaking the behaviour of an algorithm with unclear semantics.
Well, possibly next to the already existing in the ShareAdd.jl two macros: @usingany (which imports from any available environment) and @usingtmp (which creates a temporary environment), add third one @usinghere which creates or updates environment in the script’s directory, or, if this directory’s name is src, in it’s parent directory.
QuickEnv.jl v0.4.1: Now 100% Autonomous & Zero-Configuration
Following the feedback from this thread, QuickEnv v0.4.1 has been restructured and is now officially released in the General Registry.
The most important change: You no longer need to configure named environments or write special comments.
Standard Zero-Config Workflow
Simply add using QuickEnv as the first line of your script:
#!/usr/bin/env julia
using QuickEnv
using Plots, DataFrames, CSV
# Your script runs immediately...
What’s New in v0.4.1:
Zero Configuration by Default: No magic comments, no CLI flags, and no manual environment tracking required. Standard using statements are all you write. (Comments remain strictly optional for power-users who want to force specific named overrides).
Autonomous Fast Stitching (<5 ms): If your script needs Plots and DataFrames and you already have environments containing them, QuickEnv automatically synthesizes a merged Project.toml and Manifest.toml on disk in <5 ms with zero recompilation and without modifying your global @v1.x environment.
Partial Fast Stitching for Cold Starts: If new packages are needed, QuickEnv pre-stitches the known base packages and runs Pkg.add only on the missing dependencies, speeding up resolution by up to ~37% and preventing base package recompilation cascades.
Compounding Cross-Script Reuse: Every script you run enriches your local environment pool. Future scripts with overlapping dependencies launch in <1 ms with zero download, solve, or precompile overhead.
Lightweight Resource Footprint: The steady-state runtime overhead is only ~47 ms on cached runs, and each environment uses just ~3 KB of disk space (Julia stores packages and binaries globally once; environments are just tiny text pointers).
Detailed Architecture Documentation: For those interested in the underlying mechanics (why we chose manifest synthesis over LOAD_PATH stacking, the bitmask set-cover solver math, and atomic cache writes), see the newly added Design Deep-Dive and Tradeoffs Analysis.
Installation / Update:
using Pkg
Pkg.update("QuickEnv") # or Pkg.add("QuickEnv")
this is way outside my Julia skills I think, but I am just trying to comprehend how I can compare this usage with DrWatson’s @quickactivate. Once this new “fast-stiched” environment is created, is it saved as a “shared” (or “named”) environment?
Also, where does this promise of “zero recompilation” come from? How is it guaranteed? I thought Julia the language couldn’t even guarantee that. In my day to day work I find packages being recompiled all the time even when I just activate existing projects I have activated days before without ever running Pkg.add/Pkg.update. I am still baffled by why this happens and i’ve increased my precompile cache to 1000 or so
DrWatson’s @quickactivate is intended to be used inside larger projects using the project environment to run the project environment, if I understand it correctly. QuickEnv, on the other hand, is intended to work with various julia program/scripts indepndently. A lighter tool to be used whenever you just want your program/script to run, and you don’t want to think about it.
All environments created are in the shared Julia environments at ~/.julia/environments/@auto_..
Zero recompilation. Think about a big project - you created a huge environment with various packages to run the main library - with QuickEnv this environment is going to be saved somewhere (as before) - but now when you write a small program also using QuickEnv, that says uses some subset of the large environment, QuickEnv would immeidately use the old large environment (because it is a superset of what your small program need), and viola, zero compilation.
(What I understand from the AI explanations.) To avoid recompilations, you need to have Manifest.toml, not project.toml. And activating a project/directory does NOT generate a manifest! I had an interesting conversation with the AI about it, and it wrote an FAQ:
It was quite interesting, but it might be more than you want to know about the topic ;).
This sentence doesn’t make sense to me or I don’t understand it… First sentences says that “not having a manifest means no recompilations”. Second sentence says “activating a project doesn’t generate a manifest”.
Right, so we are activating projects all the time here so there should be recompilations all the time according to these statements.
Sorry. A typo, I guess. There are two files: Project.toml and Manifest.toml. Project.toml just lists the dependencies, but does not pin down the exact versions of the packages and subpackages the program depends on. This is done by Manifest.toml, which is usually not generated when you compile/run a program. You can force generating it by using the Pkg module, but that is not what Julia does by default. Activation migrates in the direct and indirect dependenciesof a program, so after you do that, the right versions are now cached in the global environment. For a while, this is fine, but then some of these subpackages drift in versions because of packages, or just get recompiled to different versions, and then when you go back to this directory/program/script, you would get recompilation. This is all explained in the FAQ…
If I understood correctly, the fast stitching generates basically a new project.toml and manifest.toml by litterally cutting out the needed bits from different sources. Or at the very least, initially bypassing the Pkg SAT solver (it’s listed as a bottleneck, so I assumed that you got rid of it).
from the design docs I see you do transitive dependency checks to make sure you don’t create an invalid environment (nice! it was my first question before reading better). Then, from what I understood given a list of packages p1,p2,... split among n manifests, you’d need to have consistency between the subtrees of every intersection. But even with no very deep subtrees (not many transitive dependencies), I’d exepect that to result in incompatibilites more often than not.
I have not used extensively enough yet to be able to say and my package use is pretty narrow. It remains to be seen how good it is. Your summary seems correct… its already making my life easier for my julia scripts…
and i’ve increased my precompile cache to 1000 or so