Hi, I am currently trying to build a docker container with Julia environment installed and I want to introduce caching to Julia package at build time. However, I didn’t find any helpful resources on whether Julia Pkg supports installing from the cache.
The cache is very similar to Python wheel tools. Whenever you tries to install some packages, it will search from cache first and then forward the request to pkg server if not found. I am wondering if Julia supports doing this and how I can use it.
I’m not 100% sure I understand what you mean, but if you want to install packages without access to the Internet, you have several options.
You can install a package directly from a path, using Pkg.add(PATH). The location of the package will be saved in Manifest.toml
All packages are cached in the depot at $JULIA_DEPOT_PATH/packages (typically ~/.julia/packages). Using using Pkg; Pkg.offline(true), you can set Pkg to offline mode, meaning it will only have access to packages in the depot, and will not attempt to update packages or otherwise access the Internet.
By default, when installing a package, Pkg will update the registry (thus connecting to the Internet), resolve the environment, and then only download packages not found in the depot. All downloaded packages are saved in the depot.
I’m not sure HOW to add packages to the depot, though. They seem to be in some randomly-generated directory name derived from the hash of the package’s content. You would need to copy the packages from an existing depot to get the right directory names.
Thanks for the response. However, I am not thinking of the offline package installation. The cache is more like a way to boost the installation speed. So even if you are online, the Julia package manager should search for the existence of a certain package locally first. If not found, then send the request to the language server for updates.
Note that Julia packages are not copied once they are downloaded: the package loader does not expect them to be collected in a folder, but instead has a set of rules to search for them in default locations (JULIA_DEPOT).
When you install a package from the register (installing from other source is a bit different) in an environment, it checks whether JULIA_DEPOT contains the requested version and download it if not present. So the second time you install, that specific version is already cached (and shared among all environments).
Actually, it is hard not to cache and share a specific version of package if you are not Pkg.developing it. Beyond that, you could also configure a central JULIA_DEPOT or some custom Pkg server.