Install package dependence from specific repository

Recently the package Keys.jl became unavailable for installation from its registered location https://github.com/bramtayl/Keys.jl, so it’s impossible to easily install packages dependent on it, e.g. JuliennedArrays. Fortunately, there is a mirror of a recent version, namely https://github.com/UnofficialJuliaMirror/Keys.jl-25c06242-cc58-593c-a204-a17bebfb33b5, and I installed the package using Pkg.add(PackageSpec(name="Keys.jl", url="https://github.com/UnofficialJuliaMirror/Keys.jl-25c06242-cc58-593c-a204-a17bebfb33b5")). However, installation of JuliennedArrays after that doesn’t find installed Keys.jl (prints ERROR: LoadError: failed to clone from https://github.com/bramtayl/Keys.jl.git, error: GitError(Code:EUSER, Class:Callback, Aborting, user cancelled credential request.)), which is due to different uuid as I understand. Consequently I try Pkg.add(PackageSpec(name="Keys.jl", uuid="25c06242-cc58-593c-a204-a17bebfb33b5", url="https://github.com/UnofficialJuliaMirror/Keys.jl-25c06242-cc58-593c-a204-a17bebfb33b5")), then the error when installing JuliennedArrays is different: ERROR: LoadError: KeyError: key "Keys" not found.

So, my question: is there any way to install the dependent package (Keys.jl) so that the installation of JuliennedArrays succeeds? I couldn’t find any.

The problem here is that there was a version 0.1.0 registered: https://github.com/JuliaLang/METADATA.jl/pull/17012 but has now been removed https://github.com/bramtayl/JuliennedArrays.jl/releases. However, that version still exists in the registry and since 0.1.0 > 0.0.3 the resolver will try to use 0.1.0 which is in fact an older (chronologically) release, and that one requires Keys.jl as a dependency (which is deleted).

So you can do add JuliennedArrays@0.0.3 for now, but we need to remove the 0.1.0 release from the registry.

The fact that we rely on package authors to be well behaved is (as is obvious here) a problem. I am quite confident that we need at some point host packages ourselves. Package authors that do bad things (either by ignorance or malice) can cause a lot of harm to the package ecosystem.

1 Like

Thanks, using version 0.0.3 seems to work. Also, orthogonal to hosting packages independent of authors, installing a dependence manually like I tried in the first post should also work, I suppose. So even if the 0.1.0 version is old, it should be possible to install.

Not if a package that that version depends on is deleted from the internet. How would it be possible to install it then?

Using a mirror, either maintained by someone else (like UnofficialJuliaMirror, see 1st post) or locally. I successfully installed Keys.jl, but installing JuliennedArrays after that gave an unhelpful error (see 1st post).

Yes, which is pretty much exactly my first post (that we need some way of hosting the packages ourselves to be resilient against package authors deleting their packages).

Allow for multiple repo urls in Package.toml by fredrikekre · Pull Request #1060 · JuliaLang/Pkg.jl · GitHub is a step in that direction.

But at least this particular dependence is already mirrored, and it didn’t help me to install a package which depends on it.

The mirror is “unofficial” and not used by the package manager. We very well could do some automatic mirroring and try fall back to that, it just currently isn’t done.

Not sure if we understand each other correctly. There is a mirror, no matter if it’s official or not - I installed the dependence (Keys.jl) from some source, be it a mirror or my local copy or something else. Then I want to install another package, which depends on Keys.jl. I think it should be possible no matter if Keys.jl is removed/mirrored officially/anything.

Yes, that should work.

Could you replicate the commands you used (from an empty environment) how you installed Keys.jl from the mirror and how to get the error when JuliennedArrays was installed.

Neither this:

Pkg.add(PackageSpec(name="Keys.jl", url="https://github.com/UnofficialJuliaMirror/Keys.jl-25c06242-cc58-593c-a204-a17bebfb33b5"))
pkg"add JuliennedArrays"

nor this:

Pkg.add(PackageSpec(name="Keys.jl", url="https://github.com/UnofficialJuliaMirror/Keys.jl-25c06242-cc58-593c-a204-a17bebfb33b5", uuid="25c06242-cc58-593c-a204-a17bebfb33b5"))
pkg"add JuliennedArrays"

works. They give different errors.

That will download the master version but you need to download the exact version that the dependency needs.

So add a rev = "65fd3d7a34ffc5d47b2a3ce11497f28f70eac4ce" keyword argument assuming that 65fd3d7a34ffc5d47b2a3ce11497f28f70eac4ce is the commit for the version that the resolver will pick (Release v0.5.0 · UnofficialJuliaMirror/Keys.jl-25c06242-cc58-593c-a204-a17bebfb33b5 · GitHub)

But this revision is the same as master, if I’m not missing something.

Trying your commands, changing name = Keys.jl to name = Keys it seems to work fine for me:

> Pkg.add(PackageSpec(name="Keys", url="https://github.com/UnofficialJuliaMirror/Keys.jl-25c06242-cc58-593c-a204-a17bebfb33b5", uuid="25c06242-cc58-593c-a204-a17bebfb33b5"))
  Updating registry at `C:\Users\Kristoffer\.julia\registries\General`
  Updating git-repo `https://github.com/JuliaRegistries/General.git`
  Updating git-repo `https://github.com/UnofficialJuliaMirror/Keys.jl-25c06242-cc58-593c-a204-a17bebfb33b5`
 Resolving package versions...
  Updating `C:\Users\Kristoffer\Keys\Project.toml`
  [25c06242] + Keys v0.5.0+ #master (https://github.com/UnofficialJuliaMirror/Keys.jl-25c06242-cc58-593c-a204-a17bebfb33b5)
  Updating `C:\Users\Kristoffer\Keys\Manifest.toml`
  [34da2185] + Compat v1.5.1
  [25c06242] + Keys v0.5.0+ #master (https://github.com/UnofficialJuliaMirror/Keys.jl-25c06242-cc58-593c-a204-a17bebfb33b5)
  [1914dd2f] + MacroTools v0.4.4
  [2a0f44e3] + Base64
  [ade2ca70] + Dates
  [8bb1440f] + DelimitedFiles
  [8ba89e20] + Distributed
  [b77e0a4c] + InteractiveUtils
  [76f85450] + LibGit2
  [8f399da3] + Libdl
  [37e2e46d] + LinearAlgebra
  [56ddb016] + Logging
  [d6f4376e] + Markdown
  [a63ad114] + Mmap
  [44cfe95a] + Pkg
  [de0858da] + Printf
  [3fa0cd96] + REPL
  [9a3f8284] + Random
  [ea8e919c] + SHA
  [9e88b42a] + Serialization
  [1a1011a3] + SharedArrays
  [6462fe0b] + Sockets
  [2f01184e] + SparseArrays
  [10745b16] + Statistics
  [8dfed614] + Test
  [cf7118a7] + UUIDs
  [4ec0a83e] + Unicode

(Keys) pkg> add JuliennedArrays
 Resolving package versions...
  Updating `C:\Users\Kristoffer\Keys\Project.toml`
  [5cadff95] + JuliennedArrays v0.1.0
  Updating `C:\Users\Kristoffer\Keys\Manifest.toml`
  [5cadff95] + JuliennedArrays v0.1.0

Oh, indeed! Thank you.