Usage of subdirectories to store multiple packages in a single repo

According to Registrator.jl#287 and Pkg.jl#1422 merged pull requests, several packages can now be organized as subdirectories of a single repo. However, I have not been able to found any detailed docs, which would explain how to use this new functionality. As a result, I would like to ask a number of questions here, on discourse.

  • How are the subdirectories supposed to be organized? Must each of the subdirectories have the normal package structure with Project.toml, Manifest.toml, ./src, ./test, ./docs etc.?
  • Is it possible to have shared docs or tests for the packages in subdirectories?
  • If the packages in subdirectories depend on each other, how imports should be organized?
  • Does subdirectory need to be registered so that other packages can use it?
  • How does one register a subdirectory package? How does one add a subdirectory package to an active environment?
  • Let’s say I want to split some core functionality of a package into a subdirectory: I have SomePackage and I want to split it into subdirectories SomePackageBase and SomePackage. Is it possible to reregister SomePackage so that it points to a subdirectory now?

The help is appreciated.

1 Like

Many/most of these questions can probably be answerd by this: Packages in subdirectories are just regular packages. They just happen to live in the same git repo.


Yes they need to be packages on their own with Project.toml, src/Package.jl entrypoint etc. Probably this structure makes the most sense:

$ tree .
.
β”œβ”€β”€ PackageA
β”‚   β”œβ”€β”€ Project.toml
β”‚   └── src
β”‚       └── PackageA.jl
└── PackageB
    β”œβ”€β”€ Project.toml
    └── src
        └── PackageB.jl

4 directories, 4 files

or perhaps

$ tree .
.
└── packages
    β”œβ”€β”€ PackageA
    β”‚   β”œβ”€β”€ Project.toml
    β”‚   └── src
    β”‚       └── PackageA.jl
    └── PackageB
        β”œβ”€β”€ Project.toml
        └── src
            └── PackageB.jl

5 directories, 4 files

Not sure what you mean – they are standalone packages. On the repo-level if you want some common tests you can structure it like this maybe:

$ tree .
.
β”œβ”€β”€ docs
β”‚   └── common_docs.html
β”œβ”€β”€ packages
β”‚   β”œβ”€β”€ PackageA
β”‚   β”‚   β”œβ”€β”€ Project.toml
β”‚   β”‚   └── src
β”‚   β”‚       └── PackageA.jl
β”‚   └── PackageB
β”‚       β”œβ”€β”€ Project.toml
β”‚       └── src
β”‚           └── PackageB.jl
└── test
    └── common_tests.jl

7 directories, 6 files

(but Pkg.jl doesn’t understand it so you must organize it yourself).

Since they are standalone package the same mechanism as if they were different repos.

Here is an example: InlineTest: bump version 0.1.0 -> 0.1.1 Β· JuliaTesting/ReTest.jl@57368e6 Β· GitHub

If it is registered just like a normal package. If you add by URL you specify the subdirectory after :: add url:/subdirpath

Yes.

1 Like

It’s probably easier to put SomePackageBase in the subdirectory, you don’t have to change as many things - especially the repository name and URL.

Docs are the thing I would definitely combine, there is not real point having separate docs, it’s probably worse for users as well.

See ModelParameters.jl for an example of this setup - a subpackage included in the main docs:

Could you please point out what is the procedure to do that.

Also, are subdirectories supported by all Julia 1.x version, or is there some lower bound? Are subdirectories supported for the LTS version of Julia?

1 Like

All Julia 1.x versions can use registered packages that live in subdirectories but complete support (Pkg.dev and similar) requires Julia 1.5 or higher.

You can find some discussion related to this in https://github.com/JuliaLang/Pkg.jl/issues/1251.

1 Like