Confused by package directory environments

I’m trying to better understand code loading, so reading that section of the manual. The manual describes package directory environments:

A package directory is a directory containing the source trees of a set of packages as subdirectories, and forms an implicit environment. If X is a subdirectory of a package directory and X/src/X.jl exists, then the package X is available in the package directory environment and X/src/X.jl is the source file by which it is loaded.

So, I create a package SharedPackage with ]generate SharedPackage. After doing so, my directory structure looks like so:

evan$ pwd
/Users/evan/jltest
evan$ ls -R
SharedPackage

./SharedPackage:
Manifest.toml	Project.toml	src

./SharedPackage/src:
SharedPackage.jl

In particular, it seems like jltest is a package directory: it contains the source tree of a package SharedPackage. In particular SharedPackage/src/SharedPackage.jl exists, as described in the manual snippet above. But I can’t load SharedPackage:

shell> pwd
/Users/evan/jltest

(v1.1) pkg> activate .

julia> using SharedPackage
ERROR: ArgumentError: Package SharedPackage not found in current path:
- Run `import Pkg; Pkg.add("SharedPackage")` to install the SharedPackage package.

Stacktrace:
 [1] require(::Module, ::Symbol) at ./loading.jl:823

The manual as quoted above seems to be saying that from the jltest environment, SharedPackage is available. But it’s not - why not? Is this not how package directory environments work / am I misreading the manual?

Could you try ]dev path/to/SharedPackage?
After developing it, it will be available through the default environment. Otherwise, you would have to be in that directory ]activate path/to/SharedPackage. If you wish to have it accessible from a non default environment, you can use ]activate path/to/TheProjectThatYouWillUse and then ]dev path/to/SharedPackage.

1 Like

Thanks! This “works” in the sense that after ]activate . and ]dev ./SharedPackage, using SharedPackage works in the jltest environment.

I’m still confused why this is necessary, as the manual says that in a directory which contains X.jl or X/src/X.jl, X is available and using-able.

The directory needs to be in LOAD_PATH.

Thanks. Can you elaborate on what LOAD_PATH should look like to make this work? The manual on code loading says:

If you want your Julia process to have access only to the packages in one project or package directory, make it the only entry in LOAD_PATH .

But:

evan$ pwd
/Users/evan/jltest
evan$ ls -R
Manifest.toml	Project.toml	SharedPackage

./SharedPackage:
Manifest.toml	Project.toml	src

./SharedPackage/src:
SharedPackage.jl
(base) Evans-MacBook-Pro:jltest evan$ julia --banner=no
julia> for _ in 1:3
       pop!(Base.LOAD_PATH)
       end

julia> push!(Base.LOAD_PATH, pwd())
1-element Array{String,1}:
 "/Users/evan/jltest"

julia> Base.LOAD_PATH
1-element Array{String,1}:
 "/Users/evan/jltest"

julia> using SharedPackage
ERROR: ArgumentError: Package SharedPackage not found in current path:
- Run `import Pkg; Pkg.add("SharedPackage")` to install the SharedPackage package.

Stacktrace:
 [1] require(::Module, ::Symbol) at ./loading.jl:823

That is, the directory jltest seems to meet the definition of a package directory (it contains SharedPackage/src/SharedPackage.jl). But setting LOAD_PATH to be just jltest doesn’t make SharedPackage available, contra the manual.

Setting Base.LOAD_PATH instead to ["/Users/evan/jltest/SharedPackage"] does make SharedPackage available, but seems to entirely miss the point of having multiple packages available in the implicit environment formed by a package directory which contains the source trees of those multiple packages.

Maybe the answer here is I am still confused, or maybe the docs need clarification? I’m afraid I’m still fundamentally unsure how implicit environments and LOAD_PATH are supposed to work.

A project will keeps track of the environment (a solution to the dependency hell). In other words, it must be aware of what code must be accessible / required. That means (1) finding where the code is, (2) getting the right version of the code, and (3) recording the information. ]add PkgName declares a package with some name as a dependency. The package manager will then look for a registry that contains the information about it such as available versions, dependencies, and where to get the code from. ]dev and ]add PkgName#branch, ]add PkgName@version work similarly. The LOAD_PATH tells the package manager where to look for that information. This is similar to having installed a program and calling it from the terminal. Just because the program/app has been installed path with the right structure is meaningless if it is not in the path (places to look for it). For example, Julia needs to be added to the path in order for applications to understand that julia means /Applications/Julia-1.2.app/Contents/Resources/julia/bin/julia. ]dev PkgName basically makes the package manager aware that the package exists and should be considered for the package functionality. When a package is registered in the default registry (General), it will get that information from the registry.

1 Like