# Could we make first-class support for packages that are "just files" and not repositories?

**URL:** <https://discourse.julialang.org/t/could-we-make-first-class-support-for-packages-that-are-just-files-and-not-repositories/36931>\
**Category:** General Usage\
**Tags:** packages, package-manager\
**Created:** [April 3, 2020, 12:59am UTC](https://discourse.julialang.org/t/could-we-make-first-class-support-for-packages-that-are-just-files-and-not-repositories/36931 "2020-04-03T00:59:17Z")\
**Posts on this page:** 1\
**Showing post:** 20

<div class="post-metadata">

**Author:** ![MA\_Laforge](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ma_laforge/32/385_2.png) [@MA\_Laforge](https://discourse.julialang.org/u/MA_Laforge)\
**Post date:** [October 14, 2021, 3:58am UTC](https://discourse.julialang.org/t/could-we-make-first-class-support-for-packages-that-are-just-files-and-not-repositories/36931/20 "2021-10-14T03:58:51Z")

</div>

I’ve had similar struggles at leveraging modules/packages in a way that makes software development simple & straightforward.

Here are some key insights that might help (using Julia v1.6.3).

### Modules: Namespaces for code organization

Julia `module`s mostly correspond to namespaces in C++ (and other languages).

- They they provide mechanisms to avoid name collisions between unrelated code.
- They allow developers to organize their code into separate logical, hierarchical units.
- They are therefore an ideal logical boundary for delimiting what code should be allowed to access the innards of a given software component & mutate its state.
- Julia `module`s therefore provide base capabilities needed to develop _conceptual_ “software modules”.
- But `module`s don’t deal with deployment issues!!!
- (Sub)-module code is directly directly “compiled” in (using `include()` when stored in separate files).
- `import` and `using` is only used to make (sub)-module code available in your namespace (it should already be “compiled”).

### Packages: A way to deploy w/dependencies & manage compile time

Julia packages provide a layer around `module`s. They help with code sharing/deployment in multiple projects:

- Due to practical requirements, packages are themselves wrapped within Julia modules.
- Packages provide Project.toml files to specify requirements (what needs to be installed/accessible).
- Packages are the minimum unit of “pre-compilation” Julia supports at this time (as far as I am aware).
- If a package can make use of fewer dependencies, it should compile faster.
- If multiple packages include the same base dependencies, “shared” code might might still need to be recompiled (as far as I know).
- External packages are loaded using either `import` or `using` - and subsequently “compiled”.

### Adding non-Git-controlled packages

Though your first instinct might be to add a local package using `pkg> add`:

```julia
julia> ]
(MyProjEnv) pkg> add /path/to/MyPackageLib/MyAwesomePkg

```

But this won’t work unless `MyAwesomePkg` is a Git repository (the Git root itself; it is insufficient to be a subdirectory of a Git repo).

- It would appear that `pkg> add` assumes the provided path is a refrence repository that is not to be altered.
- Julia’s package manager therefore _ **clones** _ this repo & generates a working snapshot of it under `~/.julia/packages`.

However, unlike `pkg> add`, `pkg> dev` simply adds packages with a direct link to the supplied path (not a clone):

```julia
julia> ]
(MyProjEnv) pkg> dev /path/to/MyPackageLib/MyAwesomePkg

```

So, this call to `pkg> dev` _ **will** _ work even if `MyAwesomePkg` is _ **not** _ a Git repository.

### Adding “single-file” packages

Unfortunately, Julia (1.6.3) won’t let you `pkg> dev` a “single-file” package:

```julia
pkg> dev /path/to/MyPackageLib/ASingleFilePkg
ERROR: Unable to parse `/path/to/MyPackageLib/ASingleFilePkg` as a package.

```

However, if you have something like:

```julia-auto
/path/to/MyPackageLib
├── MyFolderBasedPackage
| └── src
| └── MyFolderBasedPackage.jl
└── ASingleFilePkg.jl

```

Then, on startup, your project can register this library with `Base.LOAD_PATH` to access package `ASingleFilePkg`:

```julia
# Somewhere in your project's initialization code:
push!(Base.LOAD_PATH, "/path/to/MyPackageLib")

# Further on in your project:
using ASingleFilePkg #Should work

```

### Comments on `LOAD_PATH`

`LOAD_PATH` is what defines your environment stack, and therefore indirectly (1 level) adds packages to your project:

- If a directory in `LOAD_PATH` _ **does not** _ have a `Project.toml` file, Julia recognize {single .jl files} and {subdirectories having proper “package structures”} as packages.
- If a directory in `LOAD_PATH` _ **does have** _ a `Project.toml` file, Julia _ **does not** _ recognize single .jl files or subdirectories as packages. Only the packages specified in `Project.toml` are made available.

More info in Julia docs:

- [Constants · The Julia Language](https://docs.julialang.org/en/v1/base/constants/#Base.LOAD_PATH)

### Might a monolithic, multi-module package be a better solution?

```julia-auto
Aerodynamics/ # The Julia part
  .git/ #Single .git repository
  Project.toml # Dependencies for the whole of Aerodynamics.jl package
  src/
    Aerodynamics.jl
    AerodynamicsBase/ # base definitions (likely its own module)
      definitions.jl
      ...
    CommandTypes/ # Another module
      barrel_roll.jl #Seems kind of wreckless, but still...
      ...
    HugeSim/ # Directly depends on Base, but not on CommandTypes
      flightpaths.jl
      stabilization.jl
      ...
    SmallSim/
      ...

```

Code/files could be included as follows:

```julia
#src/Aerodynamics.jl
module Aerodynamics

module AerodynamicsBase
include("Base/definitions.jl")
#...

#Export symbols defined in one of the above files:
export ReallyImportantBaseType, ReallyImportantBaseFunction
end

module CommandTypes
include("CommandTypes/barrel_roll.jl")
#...
end

end #module Aerodynamics

```

Functions in the `CommandTypes` module could access types & functions using:

```julia
import ..AerodynamicsBase: SomeBaseFunction, SomeBaseType

```

And typical package usage would look something like the following:

```julia
import Aerodynamics #import: Don't pull in export-ed symbols
using Aerodynamics.AerodynamicsBase #using: DO pull in export-ed symbols

SomeBaseFunction() #Now readily available without fully-qualified-path (FQP)

```

### Monolithic, multi-module package: Principal downside

All the code in this `Aerodynamics` package needs to be compiled to use any of its (sub)-modules.

- Might be acceptable/{what you want} in an `Aerodynamics` package.
- Not like the situation where users of `Plots.jl` don’t want to pull in/“compile” ALL plotting backends when they only want/need a single one.

---

_[View the full topic](https://discourse.julialang.org/t/could-we-make-first-class-support-for-packages-that-are-just-files-and-not-repositories/36931)._