I think so, yes. For the most part though, I like using LocalRegistry.jl better than using the LOAD_PATH env variable since it helps to keep the workflow more consistent with what you’d use for package development & collaboration.
For context, I have to support two types of workflow. One inside Docker containers, and another on users machines via WSL.
My plan for this:
Dockerfiles. This is trivial. All the paths, setup, copying of folders, can be defined in the Dockerfile. It is for defining and environment after all. If I need to retain the use of config/startup.jl then I probably am forced to use JULIA_DEPOT_PATH, or both variables.
Local WSL. My intention is to export an environment variable. Probably JULIA_LOAD_PATH.
Regarding your suggestion of this LocalRegistry package - given the additional context provided do you think is it likely to be simpler solution? (I suspect you’re probably about to tell me no in this case?)
JULIA_DEPOT_PATH allows you to change the location of your .julia directory, with everything that goes there. That’s useful if you for example need to place it on a different disk.
Another use case, which is what I mostly use it for, is to simulate that you have a pristine Julia installation. Just point JULIA_DEPOT_PATH to an empty directory and you can debug package installation etc. without taking into account the specific state of your .julia directory.
It depends on how your environment looks and what kind of constraints you have in your workflows. If you have any kind of git repository hosting available the default would be store your local packages in one or more git repositories and let Pkg handle them the same way as with public packages. A local registry (which would be just another repository) is helpful to organize the local packages, especially if they have dependencies to each other.
We’re using AWS version of Github, which is extremely slow. Unfortunately this wouldn’t be practical as it would genuinely take 20-30 minutes just to clone the code.
When I distribute command line utilities written in Julia to my collaborators who don’t program in Julia, my Julia scripts are wrapped up in shell scripts which set the proper --project path as well as a dedicated JULIA_DEPOT_PATH which points to a subfolder in the folder created by decompressing my tarball. This makes sure that nothing depends on their own ~/.julia directory (if the latter exists at all). I ask them to first run a setup script which runs Pkg.instantiate() to install dependencies into JULIA_DEPOT_PATH, then run the actual utility scripts which rely on the dependencies. It’s not essential to do this, but I intended to make my code a “portable installation” which doesn’t touch anything in the user’s home directory.
Load path: a stack of environments where package identities, their dependencies, and entry points are searched for. […] The first entry is your primary environment, often the current project, while later entries provide additional packages one may want to use from the REPL or top-level scripts.
Depot: a directory on a system where various package-related resources live
Depot path: a stack of depot locations where the package manager, as well as Julia’s code loading mechanisms, look for registries, installed packages, named environments, repo clones, cached compiled package images, and configuration files. […] The first entry is the “user depot” and should be writable by and owned by the current user. The user depot is where: registries are cloned, new package versions are installed, named environments are created and updated, package repositories are cloned, newly compiled package image files are saved, log files are written, development packages are checked out by default, and global configuration data is saved. Later entries in the depot path are treated as read-only and are appropriate for registries, packages, etc. installed and managed by system administrators.
In short, the load path and depot path are orthogonal things:
When you do import Foo, the load path tells you how to decide what Foo means. It’s a list of environments to look in, and Foo is looked for in each environment. As a legacy feature, it can be a directory with a bunch of packages in it, but usually it isn’t, it’s a directory with Project.toml and Manifest.toml files in it, which is are parsed and consulted for the identity of Foo and the version of it to use. Except in the legacy case, you do not find the content of Foo in the load path.
Once you’ve identified what Foo is (resolved it to a UUID) and which version of it to use (a SHA-1 tree hash), you still need to find the actual source of that version of Foo. For this you need the depot path, which is a list of places to look for things like package contents. Once you’ve found a package, you may also need artifacts that the package uses, which can also be found in depots.
The load path is about converting a package name into a package identity (UUID) and version (SHA-1 tree hash). The depot is about storing and finding package versions and other things that you need.
The Julia equivalent of source venv/bin/activate is just
export JULIA_PROJECT=@.
The equivalent of ./venv/bin/python3 main.py is
julia --project=@.
Separate environments are built into the language instead of relying on juggling the PATH and other environment variables to simulate local environments.
If you want to automatically activate a local project environment when you are in a directory, you can use direnv (or similar). For example, a .envrc file in a project could looks like this:
This may be confusing coming from other languages since they don’t usually separate identity and content. In Python, figuring out what import foo means is inherently the same thing as finding something in a package directory with the name foo and whatever content of that directory has is the code that is used. Julia separates resolving identity and content from finding that content. This allows small, portable Project.toml and Manifest.toml files to unambiguously and reproducibly identify exactly what code should be run. You may or may not have that code, but we know what should run. For more details and motivation, you could check out this video on the design of the code loading and packaging system:
It’s not really the same. This just causes Julia to think the current working directory is always an environment directory.
If Python had a similar function (it doesn’t) then the equivalent behavior would be for Python to create venv and activate it every time the interpreter is run from a working directory which does not currently have a venv inside of it.
venv’s are relocatable, meaning you can create them anywhere, activate them from anywhere (assuming you know the path)
Doing export JULIA_PROJECT=@. is really quite a bit different to that.
Let me say that I’ve never been a fan of modifying PYTHONPATH. I’ve always considered it to be a hack.
I rarely modify it. Usually, if I am working on a project in a directory, I will develop a Python package in a pythonic style which means it can always be found from the same directory as the python3 interpreter is launched.
This works well for developing software. It doesn’t work so well once a package has matured, especially if you need to consider distributing it.
There are obviously standard ways to do this, but typically it is complicated, or at least more complicated than it needs be, I suspect.
Having a dedicated location to install packages and appending to PYTHONPATH is a reasonable solution. It’s very simple and straightforward. It doesn’t cause any problems, although people will tell you it does.
It’s not really the same. This just causes Julia to think the current working directory is always an environment directory.
You’re technically right, but it shouldn’t be difficult to see how you can use JULIA_PROJECT in a way that’s equivalent to activating a python venv. You just use export JULIA_PROJECT=/path/to/desired/project instead.
I like your suggestion - but worth pointing out that venv is mostly useful for helping to manage external dependencies (things you download with pip3). Whereas this environment variable is useful for loading local code. (Stuff you have written on the same machine.)
No, the JULIA_PROJECT affects loading of external packages installed via Pkg. This is exactly analogous to venv and pip3.
It feels a bit like you’re resisting understanding how this stuff works for the sake of complaining about it vis-à-vis Python. May I suggest that you focus more on good faith learning and less on what you wish worked exactly like Python?
