Can't Get Implicit Package Directories to Work

Tooling Package Management
1

I’m trying to create multiple packages in one repository following the implicit package directories method outlined here: Code Loading · The Julia Language

The ultimate goal is to split our large single-package codebase into multiple packages on the same monorepo, in order to reduce precompilation times which are currently ~100s with any change.

So I’ve tried this structure:

├── PackageA
│   └── src
│       └── PackageA.jl
└── TopLevel
    ├── Project.toml
    └── src
        └── TopLevel.jl

This works fine:
julia> Pkg.develop(path="TopLevel/")

But if I try to develop PackageA, I get an error message because there’s no Project.toml file:

julia> Pkg.develop(path="PackageA")
ERROR: could not find project file (Project.toml or JuliaProject.toml) in package at `PackageA` maybe `subdir` needs to be specified
Stacktrace:
  [1] pkgerror(msg::String)
    @ Pkg.Types /Applications/Julia-1.10.app/Contents/Resources/julia/share/julia/stdlib/v1.10/Pkg/src/Types.jl:70
  [2] resolve_projectfile!(env::Pkg.Types.EnvCache, pkg::Pkg.Types.PackageSpec, project_path::String)
    @ Pkg.Types /Applications/Julia-1.10.app/Contents/Resources/julia/share/julia/stdlib/v1.10/Pkg/src/Types.jl:860
  [3] handle_repo_develop!(ctx::Pkg.Types.Context, pkg::Pkg.Types.PackageSpec, shared::Bool)
    @ Pkg.Types /Applications/Julia-1.10.app/Contents/Resources/julia/share/julia/stdlib/v1.10/Pkg/src/Types.jl:629
  [4] handle_repos_develop!(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec}, shared::Bool)
    @ Pkg.Types /Applications/Julia-1.10.app/Contents/Resources/julia/share/julia/stdlib/v1.10/Pkg/src/Types.jl:705
  [5] develop(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec}; shared::Bool, preserve::Pkg.Types.PreserveLevel, platform::Base.BinaryPlatforms.Platform, kwargs::@Kwargs{io::Base.TTY})
    @ Pkg.API /Applications/Julia-1.10.app/Contents/Resources/julia/share/julia/stdlib/v1.10/Pkg/src/API.jl:214
  [6] develop(pkgs::Vector{Pkg.Types.PackageSpec}; io::Base.TTY, kwargs::@Kwargs{})
    @ Pkg.API /Applications/Julia-1.10.app/Contents/Resources/julia/share/julia/stdlib/v1.10/Pkg/src/API.jl:159
  [7] develop(pkgs::Vector{Pkg.Types.PackageSpec})
    @ Pkg.API /Applications/Julia-1.10.app/Contents/Resources/julia/share/julia/stdlib/v1.10/Pkg/src/API.jl:148
  [8] develop
    @ /Applications/Julia-1.10.app/Contents/Resources/julia/share/julia/stdlib/v1.10/Pkg/src/API.jl:146 [inlined]
  [9] develop(; name::Nothing, uuid::Nothing, version::Nothing, url::Nothing, rev::Nothing, path::String, mode::Pkg.Types.PackageMode, subdir::Nothing, kwargs::@Kwargs{})
    @ Pkg.API /Applications/Julia-1.10.app/Contents/Resources/julia/share/julia/stdlib/v1.10/Pkg/src/API.jl:176
 [10] top-level scope
    @ REPL[7]:1

This happens with every variation I could think of trying, e.g. julia> Pkg.develop(path="."; subdir="PackageA"), and activating/not activating the TopLevel project.

Am I missing something? Does develop require a Project.toml? If so, how can you use packages without a Project.toml?

2

I’m not sure that this is possible. you might have better luck giving each subpackage it’s own Project.toml, and then the top level Project.toml adds A, B, C ...

4

A package is defined by a Project.toml. There’s no package otherwise.

The documentation I linked definitely states otherwise, and gives an example of a package structure where not every package has a Project.toml. Is it just wrong/out of date?

1 Like
5

Yeah, I can get it to work with individual Project.toml files, but that means maintaining ~5 sets of dependencies, which I was hoping to avoid since these are all essentially subpackages of the same environment.

6

If the file structure works with Julia 1.0 but not with 1.10, I would suspect that the documentation is out of date and needs an issue.

1 Like
7

From the linked page:

A few general rules to note:
…
2. A package with a project file cannot depend on one without a project file since packages with project files can only load packages in graph and packages without project files do not appear in graph.

I am waiting for the upcoming workspace feature in Pkg.jl, which will hopefully simplify the organization of such mono repos.

This discussion here might also be of interest to you: Create an implicit environment with package directories

2 Likes
8

Oh, workspace looks like exactly what I want! Maybe I’ll wait until 1.12 then.

1 Like