# Seemingly incorrect dependencies resolution in Pkg

**URL:** <https://discourse.julialang.org/t/seemingly-incorrect-dependencies-resolution-in-pkg/29845>\
**Category:** General Usage\
**Tags:** package, pkg, development\
**Created:** [October 13, 2019, 1:14am UTC](https://discourse.julialang.org/t/seemingly-incorrect-dependencies-resolution-in-pkg/29845 "2019-10-13T01:14:45Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![rdyro](https://avatars.discourse-cdn.com/v4/letter/r/a587f6/32.png) [@rdyro](https://discourse.julialang.org/u/rdyro)\
**Post date:** [October 13, 2019, 1:14am UTC](https://discourse.julialang.org/t/seemingly-incorrect-dependencies-resolution-in-pkg/29845/1 "2019-10-13T01:14:45Z")

</div>

I’m trying to develop Zygote, but can’t successfully install the latest development version from Github. When I try installing the package from the Github repo directly or trying to use the dev command with it, I get the following error:

```julia
julia> versioninfo()
Julia Version 1.2.0
Commit c6da87ff4b (2019-08-20 00:03 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin18.6.0)
  CPU: Intel(R) Core(TM) i7-8559U CPU @ 2.70GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, skylake)

(v1.2) pkg> add Zygote
  Updating registry at `~/.julia/registries/General`
  Updating git-repo `https://github.com/JuliaRegistries/General.git`
 Resolving package versions...
 Installed Zygote ─ v0.3.4
  Updating `~/.julia/environments/v1.2/Project.toml`
  [e88e6eb3] + Zygote v0.3.4
  Updating `~/.julia/environments/v1.2/Manifest.toml`
 [no changes]

(v1.2) pkg> dev Zygote
  Updating git-repo `https://github.com/FluxML/Zygote.jl.git`
 Resolving package versions...
ERROR: Unsatisfiable requirements detected for package ZygoteRules [700de1a5]:
 ZygoteRules [700de1a5] log:
 ├─possible versions are: 0.1.0 or uninstalled
 └─restricted to versions 0.2.0-0.2 by Zygote [e88e6eb3] — no versions left
   └─Zygote [e88e6eb3] log:
     ├─possible versions are: 0.3.4 or uninstalled
     └─Zygote [e88e6eb3] is fixed to version 0.3.4

shell> git clone https://github.com/FluxML/Zygote.jl.git
Cloning into 'Zygote.jl'...
remote: Enumerating objects: 28, done.
remote: Counting objects: 100% (28/28), done.
remote: Compressing objects: 100% (20/20), done.
remote: Total 5709 (delta 7), reused 15 (delta 0), pack-reused 5681
Receiving objects: 100% (5709/5709), 1012.82 KiB | 4.25 MiB/s, done.
Resolving deltas: 100% (3810/3810), done.

(v1.2) pkg> add ./Zygote.jl
   Cloning git-repo `./Zygote.jl`
  Updating git-repo `/Users/ ****/Dropbox/projects/**** /Zygote.jl`
  Updating git-repo `/Users/ ****/Dropbox/projects/**** /Zygote.jl`
 Resolving package versions...
ERROR: Unsatisfiable requirements detected for package ZygoteRules [700de1a5]:
 ZygoteRules [700de1a5] log:
 ├─possible versions are: 0.1.0 or uninstalled
 └─restricted to versions 0.2.0-0.2 by Zygote [e88e6eb3] — no versions left
   └─Zygote [e88e6eb3] log:
     ├─possible versions are: 0.3.4 or uninstalled
     └─Zygote [e88e6eb3] is fixed to version 0.3.4

(v1.2) pkg> 

```

ZygoteRules is available on Github in version 0.2.0, but in the general registry only up to 0.1.0. Zygote’s Github manifest correctly specifies that ZygoteRules should be taken from a Github link and not the general registry, but the package manager does not seem to notice.

* * *

Now, maybe this isn’t the right way to go, but I tried using activate, to make sure Zygote’s manifest is used as is, but this is the result.

```julia
shell> git clone https://github.com/FluxML/Zygote.jl.git
Cloning into 'Zygote.jl'...
remote: Enumerating objects: 28, done.
remote: Counting objects: 100% (28/28), done.
remote: Compressing objects: 100% (20/20), done.
remote: Total 5709 (delta 7), reused 15 (delta 0), pack-reused 5681
Receiving objects: 100% (5709/5709), 1012.82 KiB | 3.99 MiB/s, done.
Resolving deltas: 100% (3810/3810), done.

(v1.2) pkg> add Zygote.jl/
   Cloning git-repo `Zygote.jl/`
  Updating git-repo `/Users/ ****/Dropbox/projects/**** /Zygote.jl`
  Updating git-repo `/Users/ ****/Dropbox/projects/**** /Zygote.jl`
 Resolving package versions...
   Cloning default registries into `~/.julia`
   Cloning registry from "https://github.com/JuliaRegistries/General.git"
     Added registry `General` to `~/.julia/registries/General`
ERROR: Unsatisfiable requirements detected for package ZygoteRules [700de1a5]:
 ZygoteRules [700de1a5] log:
 ├─possible versions are: 0.1.0 or uninstalled
 └─restricted to versions 0.2.0-0.2 by Zygote [e88e6eb3] — no versions left
   └─Zygote [e88e6eb3] log:
     ├─possible versions are: 0.3.4 or uninstalled
     └─Zygote [e88e6eb3] is fixed to version 0.3.4

(v1.2) pkg> activate Zygote.jl/
Activating environment at `~/Dropbox/projects/***/Zygote.jl/Project.toml`

(Zygote) pkg> add Zygote.jl/
  Updating git-repo `/Users/ ****/Dropbox/projects/**** /Zygote.jl`
  Updating git-repo `/Users/ ****/Dropbox/projects/**** /Zygote.jl`
ERROR: UndefVarError: mode not defined
Stacktrace:
 [1] #add#25(::Bool, ::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(Pkg.API.add), ::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at /Users/sabae/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.2/Pkg/src/API.jl:96
 [2] add at /Users/sabae/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.2/Pkg/src/API.jl:69 [inlined]
 [3] do_add!(::Dict{Symbol,Any}, ::Array{Pkg.Types.PackageSpec,1}, ::Dict{Symbol,Any}) at /Users/sabae/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.2/Pkg/src/REPLMode.jl:496
 [4] #invokelatest#1(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(Base.invokelatest), ::Any, ::Any, ::Vararg{Any,N} where N) at ./essentials.jl:790
 [5] invokelatest(::Any, ::Any, ::Vararg{Any,N} where N) at ./essentials.jl:789
 [6] do_cmd!(::Pkg.REPLMode.Command, ::REPL.LineEditREPL) at /Users/sabae/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.2/Pkg/src/REPLMode.jl:403
 [7] #do_cmd#23(::Bool, ::typeof(Pkg.REPLMode.do_cmd), ::REPL.LineEditREPL, ::String) at /Users/sabae/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.2/Pkg/src/REPLMode.jl:382
 [8] do_cmd at /Users/sabae/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.2/Pkg/src/REPLMode.jl:378 [inlined]
 [9] (::getfield(Pkg.REPLMode, Symbol("##28#31")){REPL.LineEditREPL,REPL.LineEdit.Prompt})(::REPL.LineEdit.MIState, ::Base.GenericIOBuffer{Array{UInt8,1}}, ::Bool) at /Users/sabae/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.2/Pkg/src/REPLMode.jl:610
 [10] #invokelatest#1 at ./essentials.jl:790 [inlined]
 [11] invokelatest at ./essentials.jl:789 [inlined]
 [12] run_interface(::REPL.Terminals.TextTerminal, ::REPL.LineEdit.ModalInterface, ::REPL.LineEdit.MIState) at /Users/sabae/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.2/REPL/src/LineEdit.jl:2306
 [13] run_frontend(::REPL.LineEditREPL, ::REPL.REPLBackendRef) at /Users/sabae/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.2/REPL/src/REPL.jl:1038
 [14] run_repl(::REPL.AbstractREPL, ::Any) at /Users/sabae/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.2/REPL/src/REPL.jl:201
 [15] (::getfield(Base, Symbol("##737#739")){Bool,Bool,Bool,Bool})(::Module) at ./client.jl:390
 [16] #invokelatest#1 at ./essentials.jl:790 [inlined]
 [17] invokelatest at ./essentials.jl:789 [inlined]
 [18] run_main_repl(::Bool, ::Bool, ::Bool, ::Bool, ::Bool) at ./client.jl:374
 [19] exec_options(::Base.JLOptions) at ./client.jl:312
 [20] _start() at ./client.jl:464

```

Am I incorrectly using the package manager or is this a dependency resolution mistake in Julia?

---

<div class="post-metadata">

**Author:** ![jling](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jling/32/212909_2.png) [@jling](https://discourse.julialang.org/u/jling)\
**Post date:** [October 13, 2019, 1:48am UTC](https://discourse.julialang.org/t/seemingly-incorrect-dependencies-resolution-in-pkg/29845/2 "2019-10-13T01:48:01Z")

</div>

I think you shouldn’t `dev` after `add`? In any case, remove any Zygote related stuff and just do `] dev Zygote` should always work.

---

<div class="post-metadata">

**Author:** ![rdyro](https://avatars.discourse-cdn.com/v4/letter/r/a587f6/32.png) [@rdyro](https://discourse.julialang.org/u/rdyro)\
**Post date:** [October 13, 2019, 1:51am UTC](https://discourse.julialang.org/t/seemingly-incorrect-dependencies-resolution-in-pkg/29845/3 "2019-10-13T01:51:38Z")

</div>

That doesn’t seem to change the behavior.

```julia
% ~ % rm -rf ~/.julia
% ~ % julia --banner=no
(v1.2) pkg> dev Zygote
   Cloning default registries into `~/.julia`
   Cloning registry from "https://github.com/JuliaRegistries/General.git"
     Added registry `General` to `~/.julia/registries/General`
   Cloning git-repo `https://github.com/FluxML/Zygote.jl.git`
  Updating git-repo `https://github.com/FluxML/Zygote.jl.git`
 Resolving package versions...
ERROR: Unsatisfiable requirements detected for package ZygoteRules [700de1a5]:
 ZygoteRules [700de1a5] log:
 ├─possible versions are: 0.1.0 or uninstalled
 └─restricted to versions 0.2.0-0.2 by Zygote [e88e6eb3] — no versions left
   └─Zygote [e88e6eb3] log:
     ├─possible versions are: 0.3.4 or uninstalled
     └─Zygote [e88e6eb3] is fixed to version 0.3.4

```

---

<div class="post-metadata">

**Author:** ![jling](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jling/32/212909_2.png) [@jling](https://discourse.julialang.org/u/jling)\
**Post date:** [October 13, 2019, 1:54am UTC](https://discourse.julialang.org/t/seemingly-incorrect-dependencies-resolution-in-pkg/29845/4 "2019-10-13T01:54:05Z")

</div>

I see, that means there’s a bug in current Zygote’s Dependency with ZygoteRules.

---

<div class="post-metadata">

**Author:** ![rdyro](https://avatars.discourse-cdn.com/v4/letter/r/a587f6/32.png) [@rdyro](https://discourse.julialang.org/u/rdyro)\
**Post date:** [October 13, 2019, 5:37am UTC](https://discourse.julialang.org/t/seemingly-incorrect-dependencies-resolution-in-pkg/29845/5 "2019-10-13T05:37:02Z")

</div>

The dependencies list is correct, I’m pretty sure. If I manually go through all the packages specified in the Manifest.toml, I can successfully add/install the Zygote package. See the script attached.

```julia
using Pkg

env = length(ARGS) > 0 ? ARGS[1] : "."
fname = length(ARGS) > 1 ? ARGS[2] : "Manifest.toml"

Pkg.activate(env)
deps = Pkg.TOML.parse(read(fname, String))

for key in keys(deps)
  dep = deps[key][1]

  spec = Dict{Symbol, String}()
  key_symb_map = Dict("repo-url" => "url", "repo-rev" => "rev")
  allowed_keys = ["name", "repo-url", "repo-rev", "version", "uuid", "path"]
  if haskey(dep, "repo-url") || haskey(dep, "path")
    allowed_keys = allowed_keys[allowed_keys .!= "version"]
  end
  for s in allowed_keys
    if haskey(dep, s)
      spec[Symbol(haskey(key_symb_map, s) ? key_symb_map[s] : s)] = dep[s]
    end
  end
  Pkg.add(PackageSpec(; spec...))
end

```

---

<div class="post-metadata">

**Author:** ![00vareladavid](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/00vareladavid/32/23521_2.png) [@00vareladavid](https://discourse.julialang.org/u/00vareladavid)\
**Post date:** [October 13, 2019, 1:35pm UTC](https://discourse.julialang.org/t/seemingly-incorrect-dependencies-resolution-in-pkg/29845/6 "2019-10-13T13:35:52Z")

</div>

You have to `add ZygoteRules#master` before you `dev Zygote`.

Dependency resolution only takes registered versions into account.

---

<div class="post-metadata">

**Author:** ![rdyro](https://avatars.discourse-cdn.com/v4/letter/r/a587f6/32.png) [@rdyro](https://discourse.julialang.org/u/rdyro)\
**Post date:** [October 13, 2019, 8:15pm UTC](https://discourse.julialang.org/t/seemingly-incorrect-dependencies-resolution-in-pkg/29845/7 "2019-10-13T20:15:11Z")

</div>

The fact that Pkg resolves dependencies of a development package using the general registry seems counter intuitive.

What if I’m developing two development packages A and B at the same time and A depends on B? Pkg will throw an error because I haven’t placed B in the general registry. The Manifest.toml of A is right there and specifies exactly where B is, yet it is being ignored.

My impression is that for development packages, especially ones specified via a git link and not from the general registry, either:

- Pkg should default to using sources specified in the Manifest
- there should be a Pkg option to explicitly use Manifest sources

If

> Dependency resolution only takes registered versions into account.

then is my script technically the solution? Isn’t there a better of way of resolving dependencies of a development package?

---

<div class="post-metadata">

**Author:** ![fredrikekre](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/fredrikekre/32/1688_2.png) [@fredrikekre](https://discourse.julialang.org/u/fredrikekre)\
**Post date:** [October 13, 2019, 9:19pm UTC](https://discourse.julialang.org/t/seemingly-incorrect-dependencies-resolution-in-pkg/29845/8 "2019-10-13T21:19:42Z")

</div>

See [Travis CI script does not clone Project.toml of dependent packages - #4 by fredrikekre](https://discourse.julialang.org/t/travis-ci-script-does-not-clone-project-toml-of-dependent-packages/29709/4)

---

<div class="post-metadata">

**Author:** ![rdyro](https://avatars.discourse-cdn.com/v4/letter/r/a587f6/32.png) [@rdyro](https://discourse.julialang.org/u/rdyro)\
**Post date:** [October 14, 2019, 4:31am UTC](https://discourse.julialang.org/t/seemingly-incorrect-dependencies-resolution-in-pkg/29845/9 "2019-10-14T04:31:20Z")

</div>

I see, so it is the case that for now the Manifest of the dependency is ignored and you have to add them manually.

I’m linking here the pull request which [fredrikekre](https://discourse.julialang.org/u/fredrikekre) mentions in the Travis post.

[https://github.com/JuliaLang/Pkg.jl/pull/1088](https://github.com/JuliaLang/Pkg.jl/pull/1088)

Thanks.