Unregistered depency not being downloaded despite being listed in Manifest.toml

Hello,

I develop a package that depends on another unregistered package. I built the Manifest.toml file to include my unregistered package (link here, the unlisted dependcy is BlockDiagonal.jl), which even shows the URL for the bitbucket repo. Oddly, though, it seems that when users try to download the repo using Pkg it doesn’t download the unregistered dependency and things fail.

Am I doing something silly here? What is the proper way for me to ask Pkg to pull the other unlisted package via a config file of some kind? Is this the intended behavior? It seems strange to me that presumably Pkg is seeing everything it needs to download the dependency but isn’t doing it.

Thank you,
Chris Geoga

Hi, bumping this question. When I try to add my package KernelMatrices.jl, which used to work fine before adding an unregistered dependency, I get the following error:

(v1.1) pkg> add git://bitbucket.org/cgeoga/KernelMatrices.jl.git #v0.5.0
  Updating git-repo `https://bitbucket.org/cgeoga/KernelMatrices.jl.git`
 Resolving package versions...
ERROR: Unsatisfiable requirements detected for package BlockDiagonal [1b959d4a]:
 BlockDiagonal [1b959d4a] log:
 ├─BlockDiagonal [1b959d4a] has no known versions!
 └─restricted to versions * by KernelMatrices [b172bcd2] — no versions left
   └─KernelMatrices [b172bcd2] log:
     ├─possible versions are: 0.5.0 or uninstalled
     └─KernelMatrices [b172bcd2] is fixed to version 0.5.0

Which seems weird to me, because the Manifest.toml file entry for BlockDiagonal.jl, my troublesome unregistered dependency, looks like this:

[[BlockDiagonal]]
deps = ["LinearAlgebra"]
git-tree-sha1 = "e844fc7e12ea0cfc6502f827764fdc1d3a07936c"
repo-rev = "0baaf631dfedb924d164b8bac4d13151d10589ee"
repo-url = "git://bitbucket.org/cgeoga/BlockDiagonal.jl.git"
uuid = "1b959d4a-82ef-11e9-2b51-1f64d30f5fc4"
version = "0.1.0"

and so it points to a version and has a git URL. I also covered my bases and made sure to tag a release with the appropriate number in BlockDiagonal.jl as well as edit the Project.toml file to look like this:

name = "BlockDiagonal"
uuid = "1b959d4a-82ef-11e9-2b51-1f64d30f5fc4"
authors = ["Chris Geoga <cgeoga@protonmail.com>"]
version = "0.1.0"

[deps]
LinearAlgebra = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e"

[compat]
julia = "1"

So I really don’t see where the issue could be here.

All of these toml files are ones that I generated using the Pkg tools and did not write by hand, so I think they are formatted correctly. So I don’t think that is the issue. I would register both of these things on METADATA, but I have the impression I can’t do that from a bitbucket repo. So I think my only option is to figure out how to make Pkg effectively pull in other unregistered dependencies.

Can somebody offer some guidance here? I am really stumped about how to do this correctly. I even tried adding a deps/build.jl file to KernelMatrices.jl that pulls in the dep, hoping that that would somehohw tell Pkg that there was extra work to be done. But clearly the failure is happening before it has a chance to see that.

How would Pkg know where to download an unregistered package? There’s an issue open about saving repo URLs in the manifest but the information currently does not exist.

The issue: https://github.com/JuliaLang/Pkg.jl/issues/492
Status: bogged down in design debate

2 Likes

Thank you for the link and reply! I think I don’t understand what the Manifest.toml file does, then. In the snippet of it I shared in the bump, it has a repo-url entry, and I would have guessed that it stored that information so that Pkg would know to pull from there.

Is there an ideal stop-gap solution you would suggest then? Should I perhaps remove the official dependency and just at a startup script that either loads it or downloads and loads it?

The manifest is not used unless the corresponding project file is the main project. See also Automatically detect unregistered dependencies by 00vareladavid · Pull Request #1088 · JuliaLang/Pkg.jl · GitHub

My understanding is: before you add KernelMatrices, you need to “manually” add its unregistered dependencies (here: Blockdiagonal). Or perhaps better develop it so that changes are tracked.

The discussion here may be useful.

Hi all—thanks for the replies. Asking users to manually install the other unregistered package is what I was hoping to avoid. Seeing that this issue is still being actively discussed and an official solution is probably a bit away, I’m just going to merge my small unregistered dependency into the bigger one. While it is obviously not the most correct thing to do, this unregistered dep is small (one file) and I can just drop it in, so I’ll do that and then revert to the “correct” thing of using it as an external unregistered dep once the dust settles.

Thanks again for the links and clarifying discussion!