More informative Project.toml or partial Manifest.toml

General Usage
,
1

In the realm of unregisted and local packages, it’s quite common to always carry around the Manifest.toml file. That is because it’s the only way to inform julia about the whereabouts of a package that is not in the public registry.

However I find this solution extreme. Manifest.toml contains a complete list for the instantiation of all involved packages and their dependencies.
But I don’t want to “freeze” all packages in the exact versions since the time I generated the Manifest.toml.

Instead I would like to cherry-pick which packages should be “frozen” and the rest be flexible.
That brings me to what I called in the title a “partial Manifest.toml” where only some packages are specified how they should be instantiated.
The same solution could be brought with a more “informative Project.toml”, that simply highlights the location for only specific packages.

Example

Let’s say my package environment is as follows:

Status `~/.julia/dev/MyPackage/Project.toml`
  [ffbed154] DocStringExtensions v0.9.3
  [8f5d6c58] EzXML v1.1.0
  [aa1b3936] GraphIO v0.6.0 
  [86223c79] Graphs v1.7.4
  [626554b9] MetaGraphs v0.7.1 `https://github.com/filchristou/MetaGraphs.jl#fixIssue29`

The versions and location of the packages are all inside the Manifest.toml
and the Project.toml contains the minimum information:

[deps]
DocStringExtensions = "ffbed154-4ef7-542d-bbb7-c09d3a79fcae"
EzXML = "8f5d6c58-4d21-5cfd-889c-e3ad7ee6a615"
GraphIO = "aa1b3936-2fda-51b9-ab35-c553d3a640a2"
Graphs = "86223c79-3864-5bf0-83f7-82e725a168b6"
MetaGraphs = "626554b9-1ddb-594c-aa3c-2596fe9399a5"

I don’t want to supply a 3rd party with just the Project.toml because MyPackage will malfunction.
And neither I want to supply a 3rd party with the Manifest.toml because all exact versions are not needed.

Is there a middle ground ?
I would like the user to see this before instantiate-ing MyPackage :

(MyPackage) pkg> st
Project MyPackage v0.1.0
Status `~/.julia/dev/MyPackage/Project.toml`
  [ffbed154] DocStringExtensions
  [8f5d6c58] EzXML
  [aa1b3936] GraphIO
  [86223c79] Graphs
  [626554b9] MetaGraphs `https://github.com/filchristou/MetaGraphs.jl#fixIssue29`

This identifies the location I want, and will directly download the latest versions.

There is a way to restrict package versions in Project.toml but not to restrict packages location (local or unregistered repo). I have the impression an addition like that would make sense (?)

PS: I know Manifest.toml doesn’t really “freeze” the packages instantiation, and they can be e.g. updated with an extra step with ]update.

4 Likes
2

There is an open issue related to that problem: Store some location info for deps in Project.toml, as well as a PR: Support [sources] section in Project.toml.

1 Like
3

One of two ways, the other being having it in another registry, e.g. one you create yourself. You can do so with my package GitHub - GunnarFarneback/LocalRegistry.jl: Create and maintain local registries for Julia packages.

3 Likes
4

how does this work with reproducibility across different parties ? I assume other people receiving the package will have to go through a set up stage to create a local registry or get access to my local registry. Or Is this automatized with some information that can be git-checked-in ?
The Manifest.toml, just needs a Pkg.update() to get another person running the package.

5

The setup stage is

using Pkg
Pkg.Registry.add("https://github.com/filchristou/FilRegistry")

or whatever URL you decide to use for your registry.

2 Likes