Pkg can't load packages present in Manifest.toml?

General Usage
1

I was playing around with Julia 1.0 and noticed a weird behavior. Given I have an environment that looks like this

(v1.0) pkg> st
    Status `~/.julia/environments/v1.0/Project.toml`
  [a81c6b42] Compose v0.6.0+ [`~/.julia/dev/Compose`]
  [e30172f5] Documenter v0.19.5
  [c91e804a] Gadfly v0.7.0

(v1.0) pkg> st -m
    Status `~/.julia/environments/v1.0/Manifest.toml`
  ...
  ...
  [5ae59095] Colors v0.9.2
  ...
  ...

I find it surprising that I can’t load Colors.jl:

julia> using Colors
ERROR: ArgumentError: Package Colors not found in current path:
- Run `Pkg.add("Colors")` to install the Colors package.

Stacktrace:
 [1] require(::Module, ::Symbol) at ./loading.jl:817

despite the fact that it’s in the Manifest and is explicitly required by Compose.jl. Instantiate and resolve don’t do anything either.

This came up when we load Colors inside the Compose docs. On Julia v0.6, it would load fine, but now I would have to explicitly install Colors despite it being required to be installed.

1 Like
2

Yes, I’m pretty sure this is the intended behavior. For a given environment, you can only load packages which are explicitly declared for that environment.

3

Adding on to that, it may be that the package that has Colors.jl as a dependency is depending on a specific version of it. This version could conflict with a version needed by another package or even the environment directly, which is why dependencies of dependencies are not automatically available.

Edit: Only half true, see below.

1 Like
4

You can only have one version of a given package at a time. You can, however, have different packages with the same name in the overall dependency graph. It is intentional that you must explicitly declare your dependencies at the top level. These are connected facts since the top-level Colors could be totally unrelated to an indirect Colors dependency.

5

You can only have one version of a given package at a time.

So the following would not be possible? Say package A depends on version 0.1 of package B, I am developing package with a dependency on version 0.2 of package B and a dependency on package A. In this scenario, package C would both directly and indirectly depend on two different versions of package B respectively.

6

Correct, that is not possible. It is quite rare for Julia packages to depend on a specific version of a dependency.

7

Hmm, that seems unfortunate, though I must admit it also seems a bit contrived as an example. What would be a good workaround should this situation show up?

8

Fix A, B or your own package so that there is a common version of A that works with both.

9

So this is a change from how things were done in the previous package manager, right? So now, even though Compose requires Colors, you need to also Pkg.add("Colors") to explicitly list it in the Project.toml

1 Like
10

Yes, precisely. It’s actually deeper than the package manager and is baked into how code loading itself works.

1 Like
11

In an emergency I guess you could make a fork of B with a different UUID and specify different versions for B and B’.

12

A fork wouldn’t even be necessary - if you’re going to fix the package anyway, you can just as well use the local branch temporarily and make a PR. After it’s merged, you can track the original package again.