METADATA vs Registry

Internals & Design
,
1

Hi all,

In the v0.6 documentation, there’s a section about maintaining a custom metadata repository. That section seems to have disappeared in the v1.0, and the Pkg.init command is not present in Pkg3. In the v1.1, this seems to be replaced by the concept of federated registries.

However, both Metadata.jl and the General Registry appear to be actively updated (and @StefanKarpinski seems to be working on syncing them right now).

So, what’s the relationship between the two concepts and what’s the current best practice for locally overriding official package dependencies?

Thanks!
Oliver

2

AFAIK at the moment the general registry is automatically synced from METADATA.jl.

It is not clear what you mean by this. If a package has certain explicit dependencies, it is very likely that changing them would break the package. You can

pkg> dev ThatPackage

and experiment though.

3

Thanks! Sorry for being vague - let me be more specific. It seems that Polyhedra.jl is compatible only with JuMP up to v0.18 based on the following error I’m getting.

(v1.1) pkg> st
    Status `~/.julia/environments/v1.1/Project.toml`
  [3391f64e] CDDLib v0.4.1
  [336ed68f] CSV v0.4.3
  [9961bab8] Cbc v0.6.0
  [861a8166] Combinatorics v0.7.0
  [a93c6f00] DataFrames v0.17.1
  [60bf3e95] GLPK v0.9.1
  [3c7084bd] GLPKMathProgInterface v0.4.4
  [4076af6c] JuMP v0.19.0+ [`~/.julia/dev/JuMP`]
  [b8f27783] MathOptInterface v0.8.3
  [fdba3010] MathProgBase v0.7.7
  [67491407] Polyhedra v0.4.5
  [295af30f] Revise v1.1.0

(v1.1) pkg> dev Polyhedra
[ Info: Path `/home/oliver/.julia/dev/Polyhedra` exists and looks like the correct package, using existing path
 Resolving package versions...
ERROR: Unsatisfiable requirements detected for package JuMP [4076af6c]:
 JuMP [4076af6c] log:
 ├─possible versions are: 0.19.0 or uninstalled
 ├─JuMP [4076af6c] is fixed to version 0.19.0+
 └─restricted to versions 0.16.0-0.18 by Polyhedra [67491407] — no versions left
   └─Polyhedra [67491407] log:
     ├─possible versions are: 0.4.5 or uninstalled
     └─Polyhedra [67491407] is fixed to version 0.4.5+

However, the REQUIRE file in Polyhedra.jl is

julia 0.7
StaticArrays 0.5
MathProgBase 0.5.10 0.8
JuMP 0.16 0.19
GeometryTypes 0.4
RecipesBase 0.2

The Polyhedra entry in the General registry seems to allow JuMP up to 0.19 as well:

oliver@oliver-acer:~/.julia/registries/General/P/Polyhedra (master=) % git remote -v
origin	https://github.com/JuliaRegistries/General.git (fetch)
origin	https://github.com/JuliaRegistries/General.git (push)
oliver@oliver-acer:~/.julia/registries/General/P/Polyhedra (master=) % git fetch
oliver@oliver-acer:~/.julia/registries/General/P/Polyhedra (master=) % cat Compat.toml
["0.0"]
GLAbstraction = "0.0-0.6"
GLVisualize = "0.0-0.7"
julia = "0.4-0.7"

["0.0-0.1"]
DataStructures = "0.2-0.15"
FixedSizeArrays = "0.0-0.2"
GeometryTypes = "0.0-0.7"

["0.1"]
julia = "0.5-0.7"

["0.1.0-0.1.2"]
MathProgBase = "0.1-0.7"

["0.1.2-0.1.3"]
JuMP = "0.1-0.19"

["0.1.3"]
MathProgBase = "0.5.10"

["0.1.4-0.2.2"]
MathProgBase = "0.5.10-0.6"

["0.1.4-0.4"]
JuMP = "0.16-0.19"

["0.2-0.3"]
julia = "0.6-0.7"

["0.2-0.4"]
GeometryTypes = "0.4-0.7"
RecipesBase = "0.2-0.6"
StaticArrays = "0.5-0.10"

["0.2.3-0.4"]
MathProgBase = "0.5.10-0.7"

["0.3"]
MultivariatePolynomials = "0.1.1-0.2"

["0.4"]
julia = "0.7-1.2"

So I’m wondering why Pkg3 thinks that Polyhedra is only compatible with JuMP <= v0.18, and how I can go about overriding that. Much of Polyhedra.jl is already compatible with JuMP v0.19, and I’m currently updating one part that’s still using the old API.

I hope that’s not too much detail :slight_smile:

Thanks!
Oliver

4

Those version ranges are inclusive of the lower bound but exclusive for the upper bound. That is, the statement in REQUIRE that reads:

JuMP 0.16 0.19

means that JuMP versions in [0.16, 0.19) are compatible. That doesn’t include 0.19.

You can find documentation for the REQUIRE syntax here: Packages · The Julia Language The asymmetrical ranges are a bit odd, but it’s really the only practical choice. If JuMP 0.16 0.19 included version 0.19, then how would you specify compatibility with any 0.18.x version but not 0.19 (since there is no way to know how many 0.18.x versions will eventually exist)? You could write 0.16 0.18.9999999 but that’s not guaranteed to work. With the specified understood to mean [0.16, 0.19) it’s easy to specify the kind of compatibility requirements you’re likely to need.

As for the 0.1-0.19 syntax in the Project.toml file, I’m not entirely sure what that does. I assume it’s the same as the REQUIRE syntax, but it’s not documented in the allowed version specifiers: 6. Compatibility · Pkg.jl

4 Likes
5

Thank you! That answers my question!