# Understanding Julia Environments - Project Environment and Package Directory

**URL:** <https://discourse.julialang.org/t/understanding-julia-environments-project-environment-and-package-directory/122082>\
**Category:** New to Julia\
**Tags:** code-organization, stacked-environments\
**Created:** [October 31, 2024, 3:06pm UTC](https://discourse.julialang.org/t/understanding-julia-environments-project-environment-and-package-directory/122082 "2024-10-31T15:06:42Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![world-peace](https://avatars.discourse-cdn.com/v4/letter/w/9f8e36/32.png) [@world-peace](https://discourse.julialang.org/u/world-peace)\
**Post date:** [October 31, 2024, 3:06pm UTC](https://discourse.julialang.org/t/understanding-julia-environments-project-environment-and-package-directory/122082/1 "2024-10-31T15:06:42Z")

</div>

I am trying to understand this documentation page which is related to code loading in Julia.

> **[Code Loading · The Julia Language](https://docs.julialang.org/en/v1/manual/code-loading/#Environments)**
>
> Documentation for The Julia Language.

Under the _Environment_ section, there are two types of environment listed.

1. Project Environment
2. Package Directory

I don’t understand what the difference between these is exactly, or why the difference might be important.

My present understanding, which may be wrong, is that a Package Directory is a directory containing a set of Project Environments.

I have come to this conclusion based on what is written here:

> 1. **A project environment** is a directory with a project file and an optional manifest file, and forms an _explicit environment_. The project file determines what the names and identities of the direct dependencies of a project are. The manifest file, if present, gives a complete dependency graph, including all direct and indirect dependencies, exact versions of each dependency, and sufficient information to locate and load the correct version.
> 2. **A package directory** is a directory containing the source trees of a set of packages as subdirectories, and forms an _implicit environment_. If `X` is a subdirectory of a package directory and `X/src/X.jl` exists, then the package `X` is available in the package directory environment and `X/src/X.jl` is the source file by which it is loaded.

My interpretation of this is that **2** is a directory containing one or more **1** ’s.

Possibly a difference might be that **packages** do not have to have the _project file_ and _manifest file_. Even if this is the case, and I’m not certain that this is correct, I don’t understand why this should be important.

Would anyone be able to explain further, possibly with an example of each structure? (The directories and files contained within each.)

---

<div class="post-metadata">

**Author:** ![Nathan\_Boyer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nathan_boyer/32/14825_2.png) [@Nathan\_Boyer](https://discourse.julialang.org/u/Nathan_Boyer)\
**Post date:** [October 31, 2024, 5:43pm UTC](https://discourse.julialang.org/t/understanding-julia-environments-project-environment-and-package-directory/122082/2 "2024-10-31T17:43:35Z")

</div>

**A project environment** just means any directory with a Project.toml file (including a package directory), and it may contain many other unrelated files too. If you `activate` this environment, then the packages listed in the Project.toml file will become available to load with `import` or `using`.

 ![image](https://global.discourse-cdn.com/julialang/original/3X/4/b/4b82b5ef677c0e8b5c41f78de6d1a97fb8f83cfd.png)

**A package directory** defines a package and contains the source code for it. If you were to download a Julia package from GitHub, the download would be a package directory. If you `activate` this environment, the packages listed in the Project.toml file become available in the same way as above. (These are the dependencies of the package.) Additionally, in this case, the package itself becomes available to load too and there is a header with package info in the Project.toml file.

 ![image](https://global.discourse-cdn.com/julialang/original/3X/1/0/105e5b2dd649b1cfd8cc454a54dbbc64a61b11fe.png)

Generally, you will be working in a standard project environment unless you are developing your own package. Note that there are also [shared environments](https://pkgdocs.julialang.org/v1/environments/#Shared-environments) (like the default global environment named with the Julia version number) that can be used to avoid saving Project.toml files in different directories.

More info [here](https://modernjuliaworkflows.org/writing/#environments).

---

<div class="post-metadata">

**Author:** ![world-peace](https://avatars.discourse-cdn.com/v4/letter/w/9f8e36/32.png) [@world-peace](https://discourse.julialang.org/u/world-peace)\
**Post date:** [October 31, 2024, 6:09pm UTC](https://discourse.julialang.org/t/understanding-julia-environments-project-environment-and-package-directory/122082/3 "2024-10-31T18:09:32Z")

</div>

> [@Nathan\_Boyer](#):
>
> **A project environment** just means any directory with a Project.toml file (including a package directory), and it may contain many other unrelated files too. If you `activate` this environment, then the packages listed in the Project.toml file will become available to load with `import` or `using`.

I see. This is very similar to Rust, which has roughly the same concept.

---

<div class="post-metadata">

**Author:** ![ffevotte](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ffevotte/32/6587_2.png) [@ffevotte](https://discourse.julialang.org/u/ffevotte)\
**Post date:** [October 31, 2024, 10:56pm UTC](https://discourse.julialang.org/t/understanding-julia-environments-project-environment-and-package-directory/122082/4 "2024-10-31T22:56:46Z")

</div>

> [@world-peace](#):
>
> My present understanding, which may be wrong, is that a Package Directory is a directory containing a set of Project Environments.

> [@world-peace](#):
>
> Possibly a difference might be that **packages** do not have to have the _project file_ and _manifest file_. Even if this is the case, and I’m not certain that this is correct, I don’t understand why this should be important.

This is all very subtle, with words like “project”, “package”, “environment” and “directory” taking slightly different meanings depending on the context. But I’d say the above two quotes are slightly incorrect. Let me try and explain my understanding.

First, I find that the [glossary](https://pkgdocs.julialang.org/v1/glossary/) in `Pkg.jl`’s documentation gives useful and precise definitions of the terms used throughout the Julia documentation. To summarize, I’d say that:

- a _project_ is any source tree following the “standard layout” and containing a `Project.toml` file declaring its dependencies. I’d say a defining feature is that a project can depend on 3rd party julia code.
- a _package_ is a particular project that can itself be used as a dependency by other project.

As such the `Project.toml` file of a _package_ needs to contain some metadata (_e.g_ a UUID) that may be missing in a _project_ that is not a package. In practice most projects actually are packages (i.e. they have their own UUID and such), even when they aren’t meant to be re-used as a dependency by other projects. This is because most projects are created by tools like `Pkg.generate` or `PkgTemplates.jl`, which will automatically take care of giving the project a name and UUID of its own.

  

I think it’s now easier to explain what goes on with environments. The two kinds of environments are referred to as “project environment” and “package directory” in the [code loading](https://docs.julialang.org/en/v1/manual/code-loading/#Environments) section of the manual, but Pkg.jl’s glossary respectively refers to them as “explicit” and “implicit” environments ; I find the latter terminology to be easier to understand.

A [“project environment”](https://docs.julialang.org/en/v1/manual/code-loading/#Project-environments) (or “explicit environment”) is an environment in which a `Project.toml` file explicitly gives the mapping between a dependency name (i.e. what you put after `using` or `import`) and a piece of code to load. (And this is true whether the `Project.toml` file defines a _project_ that is _package_ or not)

A [“package directory”](https://docs.julialang.org/en/v1/manual/code-loading/#Package-directories) (but I prefer the “implicit environment” terminology) is an environment in which the mapping between name and code is implicitly defined by the subdirectory structure, i.e. `using Foo` loads the code located in `Foo/src/Foo.jl`. Note that in this case, there may or may not exist a `Foo/Project.toml`. If there is one, it may or may not define a UUID for `Foo`. That is to say, `Foo` may or may not be a _package_.

And as you probably understood already, several environments can be combined together to form an [“environment stack”](https://docs.julialang.org/en/v1/manual/code-loading/#Environment-stacks).

> [@world-peace](#):
>
> Would anyone be able to explain further, possibly with an example of each structure? (The directories and files contained within each.)

So let’s imagine I just created a new `Foo` package that depends on `Example.jl`:

```shell
/tmp> julia -e 'import Pkg; Pkg.generate("Foo")'
  Generating project Foo:
    Foo/Project.toml
    Foo/src/Foo.jl
/tmp> cd Foo
/tmp/Foo> julia --project -e 'import Pkg; Pkg.add("Example")'
    Updating registry at `~/.julia/registries/General.toml`
   Resolving package versions...
   Installed Example ─ v0.5.5
    Updating `/tmp/Foo/Project.toml`
  [7876af07] + Example v0.5.5
    Updating `/tmp/Foo/Manifest.toml`
  [7876af07] + Example v0.5.5
Precompiling project...
  2 dependencies successfully precompiled in 1 seconds

```

Setting this project as the “home environment” gives us a **stack** with 3 environments:

```shell
/tmp/Foo> julia --project -e "foreach(println, Base.load_path())"
/tmp/Foo/Project.toml
~/.julia/environments/v1.10/Project.toml
~/.julia/juliaup/julia-1.10.6+0.x64.linux.gnu/share/julia/stdlib/v1.10

```

The first environment in the stack is **explicit** (or a “project environment”): it explicitly defines a mapping between `using Example` and the code of `Example.jl` in version 0.5.5

```shell
/tmp/Foo> tree /tmp/Foo/
/tmp/Foo/
├── Manifest.toml
├── Project.toml
└── src
    └── Foo.jl

```

Note that the fact that `Foo` itself is a full package isn’t important here, as illustrated by the second environment in the stack. This one is also **explicit** , but is a bit different in that it doesn’t adopt the “standard layout” of a full project / package: it’s merely a `Project.toml`+`Manifest.toml` pair:

```shell
/tmp> tree ~/.julia/environments/v1.10/
~/.julia/environments/v1.10/
├── Manifest.toml
└── Project.toml # <-- if you look into this you'll see it does not
                 # define a package: it has neither name nor UUID

```

Now an example of an **implicit** environment (or a “package directory”) would be the third entry in the stack: this is how the standard libraries shipped with Julia are organized. In this environment, `using ArgTools` loads the code in `ArgTools/src/ArgTools.jl`

```shell
/tmp> tree ~/.julia/juliaup/julia-1.10.6+0.x64.linux.gnu/share/julia/stdlib/v1.10
~/.julia/juliaup/julia-1.10.6+0.x64.linux.gnu/share/julia/stdlib/v1.10
├── ArgTools
│ ├── Project.toml # <-- It so happens that there is a complete Project.toml
│ └── src # with a UUID, but it could as well not be here
│ └── ArgTools.jl
├── Artifacts
│ ├── Project.toml
│ └── src
│ └── Artifacts.jl
[...]

```

---

<div class="post-metadata">

**Author:** ![ffevotte](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ffevotte/32/6587_2.png) [@ffevotte](https://discourse.julialang.org/u/ffevotte)\
**Post date:** [October 31, 2024, 11:13pm UTC](https://discourse.julialang.org/t/understanding-julia-environments-project-environment-and-package-directory/122082/5 "2024-10-31T23:13:33Z")

</div>

Oh and maybe a word of caution: I’d say it’s not terribly popular these days for normal Julia users to create “package directories”.

If you develop a bunch of libraries that are meant to be used in another project, I’d say the recommended way would be to structure each library to be a real package, and then to `Pkg.develop` each one of them into the project that needs to use them. This way you only deal with explicit environments and explicitly declared dependencies. (Rather than putting all libraries in the same top directory and tweaking `LOAD_PATH` to add this directory to the stack)

---

<div class="post-metadata">

**Author:** ![world-peace](https://avatars.discourse-cdn.com/v4/letter/w/9f8e36/32.png) [@world-peace](https://discourse.julialang.org/u/world-peace)\
**Post date:** [November 1, 2024, 11:52am UTC](https://discourse.julialang.org/t/understanding-julia-environments-project-environment-and-package-directory/122082/6 "2024-11-01T11:52:37Z")

</div>

> [@ffevotte](#):
>
> A [“project environment”](https://docs.julialang.org/en/v1/manual/code-loading/#Project-environments) (or “explicit environment”) is an environment in which a `Project.toml` file explicitly gives the mapping between a dependency name (i.e. what you put after `using` or `import`) and a piece of code to load. (And this is true whether the `Project.toml` file defines a _project_ that is _package_ or not)
> 
> A [“package directory”](https://docs.julialang.org/en/v1/manual/code-loading/#Package-directories) (but I prefer the “implicit environment” terminology) is an environment in which the mapping between name and code is implicitly defined by the subdirectory structure, i.e. `using Foo` loads the code located in `Foo/src/Foo.jl`. Note that in this case, there may or may not exist a `Foo/Project.toml`. If there is one, it may or may not define a UUID for `Foo`. That is to say, `Foo` may or may not be a _package_.

Ah I’ve just realized. This means if you have a _subdirectory_ called `DataFrames` this could potentially conflict with the package `DataFrames` which would otherwise be downloaded from the `pkg` repository.

Which one takes precidence?

---

<div class="post-metadata">

**Author:** ![ffevotte](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ffevotte/32/6587_2.png) [@ffevotte](https://discourse.julialang.org/u/ffevotte)\
**Post date:** [November 1, 2024, 2:14pm UTC](https://discourse.julialang.org/t/understanding-julia-environments-project-environment-and-package-directory/122082/7 "2024-11-01T14:14:55Z")

</div>

> [@world-peace](#):
>
> Ah I’ve just realized. This means if you have a _subdirectory_ called `DataFrames` this could potentially conflict with the package `DataFrames`

Yes, that’s the idea with “package directories”: everything is determined by the subdirectory name, there is no attempt at disambiguating identical names with UUIDs or choosing a specific version.

> [@world-peace](#):
>
> Which one takes precidence?

I think the first environment in the stack that knows about the name you provided.

---

<div class="post-metadata">

**Author:** ![world-peace](https://avatars.discourse-cdn.com/v4/letter/w/9f8e36/32.png) [@world-peace](https://discourse.julialang.org/u/world-peace)\
**Post date:** [November 1, 2024, 2:32pm UTC](https://discourse.julialang.org/t/understanding-julia-environments-project-environment-and-package-directory/122082/8 "2024-11-01T14:32:38Z")

</div>

> [@ffevotte](#):
>
> I think the first environment in the stack that knows about the name you provided.

That would be the _local subfolder_ rather than the `DataFrames` which gets _pulled from the internet_ by `pkg`?

---

<div class="post-metadata">

**Author:** ![ffevotte](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ffevotte/32/6587_2.png) [@ffevotte](https://discourse.julialang.org/u/ffevotte)\
**Post date:** [November 1, 2024, 2:52pm UTC](https://discourse.julialang.org/t/understanding-julia-environments-project-environment-and-package-directory/122082/9 "2024-11-01T14:52:37Z")

</div>

To be clear: if you have

1. `/path1/Project.toml` containing a `[deps]` line referring to `DataFrames` (i.e. an explicit environment knowing about `DataFrames`), and
2. `/path2` containing a `DataFrames/src/DataFrames.jl` source file (i.e. an implicit environment also knowing about the `DataFrames` name),

then

```julia-repl
julia> LOAD_PATH
3-element Vector{String}:
 "/path1/Project.toml"
 "/path2"
 "@stdlib"
julia> using DataFrames

```

will refer to `DataFrames` coming from the registry, downloaded by `Pkg` in the version determined by `/path1/Manifest.toml`, and automatically stored somewhere under the `~/.julia` directory.

Whereas

```julia-repl
julia> LOAD_PATH
3-element Vector{String}:
 "/path2"
 "/path1/Project.toml"
 "@stdlib"
julia> using DataFrames

```

will refer to whatever happens to be in `/path2/DataFrames/src/DataFrames.jl`.

---

<div class="post-metadata">

**Author:** ![world-peace](https://avatars.discourse-cdn.com/v4/letter/w/9f8e36/32.png) [@world-peace](https://discourse.julialang.org/u/world-peace)\
**Post date:** [November 1, 2024, 2:59pm UTC](https://discourse.julialang.org/t/understanding-julia-environments-project-environment-and-package-directory/122082/10 "2024-11-01T14:59:13Z")

</div>

Right - it’s not as simple as I thought. Thanks, I will need to think on it a bit, probably