Workspaces + Monorepo of unregistered packages

[nandoconde]

Hello! I was reading the Julia v1.12 release blog post and read about the use of the workspace feature and how it can help with monorepos.

Right now, I have several unregistered interdependent packages and the approach I was following is having a LocalRegistry where those are registered, so I can do things like (pkg)> add MyLocalPackage1 and (pkg)> update.

How could I keep everything together in a single root directory that I can commit to version control and still keep the interdependencies between them using this new feature?

Thanks!

[simsurace]

I also don’t understand how the new [workspaces] feature is supposed to work. Let’s say I have a monorepo with a bunch of packages.

1. Shall I create a project at the repo root that includes these packages in [workspace]? What if anything is different from just listing them as [deps] in this root project and specifing [sources] for them?
2. When working inside one of the packages, how will the dependencies be resolved? I think the documentation suggests that somehow the resolver looks if there are any workspaces that include this package. However, from my first tests it seems as if the local Manifest is still being used.

[icweaver]

Thanks for opening this! I would also like to learn more about the connection between [workspace] and [sources]

[icweaver]

These updates to the Pkg manual really made it click for me. Only needing to work with a single, unified Manifest now feels sooo slick. Thanks again for this awesome functionality!

update docs to show how workspaces are used for test deps

[langestefan]

If I understand correctly, if I already have a top-level project/Project.toml and both project/test/Project.toml, project/docs/Project.toml exist, I can remove the Manifest.toml files in project/docs and test/docs after adding:

[workspace]
projects = ["test", "docs"]

To the top-level project/Project.toml. Sounds like this should be the default way of managing your project?

[icweaver]

That’s my understanding. I really like the updated pkg prompt that shows this now, e.g.,

(MyPkg/test) pkg>

Another perk of this unified structure, IIUC, is that we don’t need to sprinkle MyPkg = {path = ".."} in the [sources] section of the Project.toml of our various subdirs now if we want our docs or test envs, for example, to pick up our package. I basically never need to reach for this or deving my packages anymore

[langestefan]

It’s very clean! I proposed to add it to BestieTemplate.jl: Usage of `[workspace]` section in `Project.toml` · Issue #559 · JuliaBesties/BestieTemplate.jl · GitHub

[csvance]

Just tested and it works for Revise too! Of course if you actually need to call and interact with the workspace packages directly you still need to dev them unless you are ok typing out TopLevelPackage.WorkspacePackage.$name every time.

There were many great changes in Julia 1.12 but I’m convinced the workspace feature was one of the best. At least for modular monoliths, we get the best of all worlds without any compromises. Precompile can run in parallel for many of the workspace packages. Only having one repository to change means you can do all of your testing for each pull request, eliminating the possibility of a change to one repo silently breaking things in another that depends on it without far more custom tooling.

[Tamas_Papp]

Clarifying question: should child repos (eg test/) dev .. the parent? It seems to be necessary, otherwise it is not found.

[Mason]

No, they should find the parent automatically. They just need the UUID

[Tamas_Papp]

Then I am doing something wrong. Consider this MWE (all files below):

tamas@tamas-laptop /tmp/w % julia --project=/tmp/w
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.12.1 (2025-10-17)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org release
|__/                   |

(w) pkg> st
Project w v0.0.0
Status `/tmp/w/Project.toml`

shell> cat Project.toml
name = "w"
uuid = "79ea39c1-c3c5-4ec3-b3d4-63920945435d"

[workspace]
projects = ["test"]

(w) pkg> activate test/
  Activating project at `/tmp/w/test`

(w/test) pkg> st
Status `/tmp/w/test/Project.toml` (empty project)

julia> using w
ERROR: ArgumentError: Package w not found in current path.
- Run `import Pkg; Pkg.add("w")` to install the w package.
Stacktrace:
 [1] macro expansion
   @ ./loading.jl:2375 [inlined]
 [2] macro expansion
   @ ./lock.jl:376 [inlined]
 [3] __require(into::Module, mod::Symbol)
   @ Base ./loading.jl:2358
 [4] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:2334
 [5] top-level scope
   @ REPL[4]:1

(w/test) pkg> activate .
  Activating project at `/tmp/w`

julia> using w
Precompiling w finished.
  1 dependency successfully precompiled in 1 seconds

My expectation was that from w/test, w would be just found.

The files (sorry, could not attach a zip)

tamas@tamas-laptop /tmp/w % tree
.
├── Manifest.toml
├── Project.toml
├── src
│   └── w.jl
└── test
    ├── Project.toml
    └── runtests.jl

3 directories, 5 files
tamas@tamas-laptop /tmp/w % cat Manifest.toml 
# This file is machine-generated - editing it directly is not advised

julia_version = "1.12.1"
manifest_format = "2.0"
project_hash = "117bf6d011a054f6ee52fed5d22343ac64040d16"

[[deps.w]]
path = "."
uuid = "79ea39c1-c3c5-4ec3-b3d4-63920945435d"
version = "0.0.0"
tamas@tamas-laptop /tmp/w % cat Project.toml 
name = "w"
uuid = "79ea39c1-c3c5-4ec3-b3d4-63920945435d"

[workspace]
projects = ["test"]
tamas@tamas-laptop /tmp/w % cat src/w.jl 
module w
end
tamas@tamas-laptop /tmp/w % cat test/Project.toml 
tamas@tamas-laptop /tmp/w % cat test/runtests.jl 
using w

[Mason]

Hm, this might be a bug with Pkg.add then, i cant check locally right now, but does it work if you add the lines

[deps]
w = "79ea39c1-c3c5-4ec3-b3d4-63920945435d"

To w/test/Project.toml and then instantiate?

[Tamas_Papp]

Trying to pkg> add it fails,

(w/test) pkg> add w
ERROR: The following package names could not be resolved:
 * w (79ea39c1-c3c5-4ec3-b3d4-63920945435d in manifest but not in project)

but manually adding it, as you suggested, does work. But only because it is already in the Manifest.toml* since I dev’d it before. No dev, it is not found.

Self-contained script:

(@v1.12) pkg> generate mwe
  Generating  project mwe:
    mwe/Project.toml
    mwe/src/mwe.jl

(@v1.12) pkg> activate mwe
  Activating project at `/tmp/mwe`

shell> cd mwe
/tmp/mwe

julia> open(io -> print(io, "\n\n[workspace]\nprojects = [\"test\"]\n"), "Project.toml", "a")

shell> mkdir test

(mwe) pkg> activate test
  Activating new project at `/tmp/mwe/test`

(mwe/test) pkg> add mwe
ERROR: The following package names could not be resolved:
 * mwe (not found in project, manifest or registry)

[csvance]

It’s also an issue the other way around. I have to run Pkg.develop(path="path/to/MyWorkspacePackage.jl") to add a workspace package as a dependency to the top level project. It’s not enough to just do something like this in Project.toml:

name = "MyTopLevelPackage"
...

[workspace]
project = ["path/to/MyWorkspacePackage.jl"]

and then do pkg> add MyWorkspacePackage.

ERROR: the following package names could not be resolved:
 * MyWorkspacePackage (not found in project, manifest or registry)

Besides the annoyance of having to manually dev a ton of packages in a specific order for many different workspace packages, I managed to split up my main project into 11 different workspace packages and things are working correctly. Precompile is way faster too now that many of the packages can be compiled at the same time.

[simsurace]

I‘m not sure that was the intended use case. If package A depends on B, why do you want the Manifest for A to be stored in B?

EDIT: I think I got confused myself, lol

[csvance]

The confusion is likely coming from people discussing using the workspace features in two different ways in a single thread.

Let A be the top level package which has a workspace.

#1: Test / Docs

Let B be a test / docs package with additional dependencies we don’t want to include in A. B needs to import A to do tests / generate documentation.

#2: Mono Repo

Let B be a package providing functionality needed by A. A needs to import one or more B style packages in order to compose together packages at the highest level.

I could be wrong, but I think that previously every B needed to be its own Git repository, but now thanks to the workspace feature you only need a single Git repository for A which also contains all B style packages.

[BeastyBlacksmith]

Does this allow for running pkg> dev A and it works even when B is unregistered?

[simsurace]

But wait, that was what [sources] was for. [sources] is for finding dependencies when they are not registered. [workspace] is not for that, it is for coupling the resolving process (Manifest file) for different packages even if they are not dependencies of each other.

[csvance]

@simsurace

But wait, that was what [sources] was for.

I had gotten the impression from the Julia 1.12 release notes that the [workspace] approach was superseding the [sources] based approach at least for testing:

Test-specific dependencies are now recommended to be specified using the workspace approach (a project file in the test directory that is part of the workspace defined by the package project file).

I’m confused as to whether its intended to combine [workspace] with [sources].

[simsurace]

I believe that regarding testing, the [workspace] approach supersedes the [extras] approach which is now deprecated. I’ve never seen [sources] recommended for testing. [sources] is useful when you are depending on other packages that are not registered. Of course the test environment depends on the parent package to be tested and that may not be registered, but in my understanding that’s always been special cased to find the parent package regardless of whether it is registered.

For testing, instead of [extras] and [targets] it has also been possible for some time (Julia 1.6 maybe) to have a separate test/Project.toml that lists all test dependencies, and does not need to list the parent package, which was implicit. This continues to work AFAIK,

What is new (and this is my current understanding which is still evolving) is that you can now have this test/Project.toml file, and then define a workspace in the parent package that includes this test folder. Now you must also list the parent package in test/Project.toml, and what you get out of this setup (which is optional) is that now the tests will run with the dependencies as resolved in the parent package (or recursively packages up the file tree if the parent package itself is part of a workspace), as recorded in the root manifest file, whereas previously, the test environment would have been resolved independently, with maybe quite different versions than what you would get if you just installed your package by itself.

From the perspective of a monorepo use case, I think you’ll want to make sure that you are testing with some overall manifest that is the same as what you use in production, so that’s where the [workspace] feature really shines.

I’m also still trying and figuring this out, so anyone please feel free to correct me if I’m wrong.