Where does Pkg(3) find version numbers?

Internals & Design
1

I am trying to install… or register, or record a package with a particular version number via (v0.7) pkg> dev packagepath, but am unable to do this. The package is LambertW, which is registered. Pkg will only add it to Project.toml with a very old version number. I have tried both local repos and the registered package on github. Here is an example:

(v0.7) pkg> preview dev /home/lapeyre/.julia/dev/LambertW#master
───── Preview mode ─────
┌ Warning: packages will need to have a [Julia]Project.toml file in the future
└ @ Pkg.Types Types.jl:603
 Resolving package versions...
 Installed Revise ─ v0.5.4
 Installed Arpack ─ v0.1.4
  Updating `~/.julia/environments/v0.7/Project.toml`
  [e24f45a5] ? InverseLaplace v0.0.3+ [`~/.julia/dev/InverseLaplace`] ⇒ v0.2.0+ [`~/.julia/dev/InverseLaplace`]
  [984bce1d] + LambertW v0.2.0+ [`~/.julia/dev/LambertW`]
  Updating `~/.julia/environments/v0.7/Manifest.toml`
  [e24f45a5] ? InverseLaplace v0.0.3+ [`~/.julia/dev/InverseLaplace`] ⇒ v0.2.0+ [`~/.julia/dev/InverseLaplace`]
  [984bce1d] + LambertW v0.2.0+ [`~/.julia/dev/LambertW`]
  Building skipping building in preview mode

The master branch (which I specify explicitly) is tagged v0.4.0. The tag v0.2.0 is a couple of years old. The latest version here

~/.julia/v0.7/METADATA/LambertW/versions/

is v0.2.0. But, I’d like to ignore this, and just use a local package, for now. This is actually holding up development on another package that depends on LambertW v0.4.0. I am using a REQUIRE file for this. Maybe I have to switch to Project.toml in order to specify a non-registered location. In any case this dependency has been removed, and will stay that way until I am able to “develop” LambertW v0.4.0.

2

Unless there is a Project.toml file it uses the latest version in METADATA.

If there is a Project.toml file, it looks at e.g. the version = "0.1.0" field in that file.

3

Ok. For the record; In the end, it seemed fastest to do the following. Add this to .julia/environments/v0.7/Manifest.toml:

[[LambertW]]
deps = ["Test"]
git-tree-sha1 = "73db9837dfa9c4f2a0793ee68451393fb7f03339"
uuid = "0b6a3af8-85db-11e8-03ee-b981167d19f2"
path = "/home/lapeyre/.julia/dev/LambertW"
version = "0.4.0"

with a generated uuid, and the desired commit (with a tag that matches the version). Then this in .julia/environments/v0.7/Project.toml:

LambertW = "0b6a3af8-85db-11e8-03ee-b981167d19f2"

Then pkg > status shows a harmonious state.
For the package depending on LambertW, I will try a Project.toml.

4

If there is no version explicitly in the file, it’s using the UUID? The UUID’s are for specific versions right?

5

If I delete the version line in Manifest.toml, I get this:

(v0.7) pkg> preview resolve 
───── Preview mode ─────
ERROR: KeyError: key "version" not found
6

I know that version is always in Manifest.toml, but it was my understanding that only Project.toml is mandatory, and in that one usually there is no explicit version.

Could you guys also clarify whether Manifest.toml should be included in repos?

7

Modifying the manifest by hand is usually not a good idea. It will likely be overwritten quickly.

No, it is for specific packages.

See Does Manifest.toml belong in the repository?

1 Like
8

Interesting. So if I only have a Project.toml, I’ll always get the latest tagged versions of everything?

That would actually explain a lot of things I have been confused about.

1 Like
9

Not sure what the question here is, could you elaborate a bit?

10

I have wondered about this too, so this is how I would phrase the question: suppose I install a package (with add or dev) that has a Project.toml but no Manifest.toml, how are the versions that are used to populate Manifest.toml determined? Is it the “latest” feasible solution (not sure that is well-defined, but least up to a partial ordering), or just “a” feasible solution?

1 Like
11

The resolver tries to find the latest versions of packages that fits within the constraints given by the compat section of the project.

It is not any different from what the old package manager did.

2 Likes
12

Ohhhh, that clarifies, soooo much thanks! I think that has been my main source of confusion about this whole thing, it must not have been in the docs at the time when I read them. Anyway, I’m really liking this setup! In particular, I like that it defaults to getting most recent tagged versions of everything. Looking forward to using it more now that I understand what’s going on.

13

I agree its not a good idea. But, I could find no other way to work with the version I want.

In any case. My solution above does not work:

(v0.7) pkg> preview resolve
───── Preview mode ─────
 Resolving package versions...
ERROR: Unsatisfiable requirements detected for package LambertW [0b6a3af8]:
 LambertW [0b6a3af8] log:
 ├─possible versions are: 0.0.0 or uninstalled
 ├─LambertW [0b6a3af8] is fixed to version 0.0.0
 └─restricted to versions 0.4.0 by an explicit requirement — no versions left

I’ll think I’ll wait until version 0.4.0 is in METADATA.

I don’t know exactly what “package” means here.

14

Every package (something you can load with using) is identified by a unique UUID.

You could perhas relax the version constraint on LambertW?

15

I’ll try that. It’s not urgent. I have to wait anyway till v0.4.0 is in METADATA in order for automatic CI to work.

16

I keep getting errors like └─restricted to versions 0.0.0 by an explicit requirement — no versions left when trying to dev my own packages. I don’t know why. I can’t find any Manifest.toml files that say 0.0.0.

The problem is I have no idea how to actually fix anything when something goes wrong, so I feel like I’m forced to delete everything and start over whenever there’s a problem.

Maybe I can try and edit all the instances of “0.1.0” I find for that package, replacing them with “0.0.0”, and hope that works?

17

This typically means you have an unregistered package without a project file or a project file without a version entry.

If you have examples that fail, don’t be afraid of opening issues on the Pkg repo. Pkg is new, so better documentation and error messages etc are needed but those are much easier to add based on feedback from users than just coming up with examples in your head.

1 Like
18

EDIT: Somehow switched tabs without noticing. Was filing an issue on Github.

Was JuliaLang’s github the correct place? Pkg3 -- resolve fails with unsatisfiable restricted versions on `dev` · Issue #28639 · JuliaLang/julia · GitHub

Or is there still another active repository, outside of the stdlib location?
https://github.com/JuliaLang/julia/tree/master/stdlib/Pkg

19

Thanks for the report. This seems to be a stupid bug in Pkg. https://github.com/JuliaLang/Pkg.jl/pull/646 should fix it.

(Pkg) pkg> dev https://github.com/chriselrod/TriangularMatrices.jl
  Updating registry at `~/.julia/registries/General`
  Updating git-repo `https://github.com/JuliaRegistries/General.git`
  Updating git-repo `https://github.com/chriselrod/TriangularMatrices.jl`
[ Info: Path `/Users/kristoffer/.julia/dev/TriangularMatrices` exists and looks like the correct package, using existing path instead of cloning
 Resolving package versions...
ERROR: Unsatisfiable requirements detected for package TriangularMatrices [5ebe76dc]:
 TriangularMatrices [5ebe76dc] log:
 ├─possible versions are: 0.0.0 or uninstalled
 ├─TriangularMatrices [5ebe76dc] is fixed to version 0.0.0
 └─restricted to versions 0.1.0 by an explicit requirement — no versions left

julia> import Pkg # Load Pkg with fix

(Pkg) pkg> dev https://github.com/chriselrod/TriangularMatrices.jl
  Updating registry at `~/.julia/registries/General`
  Updating git-repo `https://github.com/JuliaRegistries/General.git`
  Updating git-repo `https://github.com/chriselrod/TriangularMatrices.jl`
[ Info: Path `/Users/kristoffer/.julia/dev/TriangularMatrices` exists and looks like the correct package, using existing path instead of cloning
 Resolving package versions...
  Updating `Project.toml`
  [5ebe76dc] + TriangularMatrices v0.0.0 [`~/.julia/dev/TriangularMatrices`]

Edit: The PR needs some fixup but it is mostly correct. At least I know the problem.

1 Like
20

Saw tests failed for today. Once they’re fixed and fixes are merged in Pkg master, what’s the easiest way to check them out locally? I know we don’t need the same Julia and standard lib versions. My first try was simply:

(v1.0) pkg> add Pkg#master
  Updating registry at `~/.julia/registries/General`
  Updating git-repo `https://github.com/JuliaRegistries/General.git`
ERROR: The following package names could not be resolved:
 * Pkg (44cfe95a-1eb2-52ea-b672-e2afdf69b78f in manifest but not in project)
Please specify by known `name=uuid`.

Would also be nice to get the startup.jl fix too.
https://github.com/JuliaLang/Pkg.jl/issues/609