While trying to upgrade a package I found that some other package held it back with an upper version bound. To track down the offending package, I found the following function useful
function find_holdback(held_back_package, recursive = 0, installed = collect(keys(Pkg.installed())), inset=0)
@assert held_back_package ∈ installed "The package you are querying is not installed"
for package in installed
REQUIRE = Pkg.dir(package, "REQUIRE")
isfile(REQUIRE) || continue
for line in eachline(REQUIRE)
splitline = split(line, ' ')
splitline[1] == held_back_package || continue
for i = 1:inset print(" ") end
if length(splitline) > 2
print_with_color(:red, @sprintf("Package %-35s requires %s\n", package, line))
else
@printf("Package %-35s requires %s\n", package, line)
end
recursive > 0 && find_holdback(package, recursive-1, installed, inset+1)
end
end
end
julia> find_holdback("ForwardDiff", 2) # Prints requires with a depth of 2
I like this, but IIUC your approach is not quite right when an upper bound is introduced via METADATA. This occurred, for example, when the recent breaking revision of DataFrames forced them to bound all dependencies. You could handle these cases by getting the lines from METADATA/$package/versions/$version/requires file instead of the package’s REQUIRE file.
This looks like a really handy function, but it doesn’t seem to be helping me - I wonder if anyone can see what’s going on? I want the latest version of RCall (0.9.0, soon to be 0.10.0), but it’ll only give me 0.8.1. Installing RCall, I see:
julia> Pkg.checkout("RCall")
INFO: Checking out RCall master...
INFO: Pulling RCall latest master...
ERROR: resolve is unable to satisfy package requirements.
The problem was detected when trying to find a feasible version
for package CategoricalArrays.
However, this only means that package CategoricalArrays is involved in an
unsatisfiable or difficult dependency relation, and the root of
the problem may be elsewhere.
Stacktrace:
[1] resolve(::Dict{String,Base.Pkg.Types.VersionSet}, ::Dict{String,Dict{VersionNumber,Base.Pkg.Types.Available}}) at ./pkg/resolve.jl:48
...
Suggesting there’s some problem with dependencies for CategoricalArrays, but when I use find_holdback() again, I just get:
And nothing has an upper bound on CategoricalArrays. In fact searching through the REQUIRE files, I see almost nothing with an upper bound on anything any longer (LearnBase and IntervalTrees for some reason). Any suggestions?
Thanks. According to my .julia/v0.6 folder, DataTables has a pretty simple:
CategoricalArrays 0.1.2
requirement with no upper limit (as suggested by find_holdback()). However, I’ve just deleted the whole folder and reinstalled some things and the problem seems to have gone away. I haven’t reinstalled everything, but touch wood the problem has gone away…
I think the problem is that the package manager uses the information present in METADATA, which doesn’t necessarily reflect the one returned by the script, which looks at each package’s directory. In the present case, we have added upper bounds to METADATA, and updated DataTables master, but no release has been tagged.
Ah, okay. Perfect. Adding DataTables makes everything fall apart. That clarifies everything. I’d forgotten that METADATA can be changed without changing the underlying packages.
Is there something similar that can be used with a recent version of Julia? I my case I would like to update CSV/DataFrames, but not sure what is holding those packages back.
I was greatly interested in this find_holdback. I don’t understand the claim that it is now “built” into the package manager. Please forgive me for bumping this topic. How do you get a list of packages and their “requires”? Here’s the best I could find:
# Query package information as seen on https://github.com/invenia/JLSO.jl/issues/32
deps = Pkg.dependencies()
# Dict{Base.UUID, Pkg.Types.PackageInfo}
for (key, value) in deps
info = deps[key]
for fname in fieldnames(typeof(info))
println(fname,": ", getfield(info, fname))
end
end
which yields a gigantic list of elements like this one (the first one in my case):
By the way another way to request a specific package version (I personally don’t use ]? because I don’t know how to exit the Pkg prompt to go back to the Julia prompt ):
However, this will not help discover a compatibility problem. And Julia will not obey the command if there’s an incompatibility, as far as I could tell. You cannot force a version if it’s not compatible. You cannot “push” incompatibility issues to another package. As far as I can tell in my limited experience.
P.S. A couple of things are broken with find_holdback. Is it beyond repair? Pkg.installed is now Pkg.dependencies and I think you need using Printf too.
What do you mean by “Julia will not obey the command of there’s an incompatibility”? Presumably this means that it will raise an error, which will then tell you what the incompatibility is?
Also you get out of the package REPL mode (like all other REPL modes AFAIK) by pressing backspace.
What do you mean by “Julia will not obey the command if there’s an incompatibility”?
All I mean is that after I identified that ImplicitPlots was holding back StaticArrays to some paleolithic version, I did Pkg.rm("ImplicitPlots") followed by Pkg.update("StaticArrays"), but “Julia did not obey” in the sense that I was still stuck with the super old version of StaticArrays:
Updating registry at `~/.julia/registries/General`
No Changes to `~/Julia/workspace/Project.toml`
No Changes to `~/Julia/workspace/Manifest.toml`
Deleting all the packages listed in the Project.toml and Manifest.toml (and additionally manually removing the old version of StaticArrays out of superstiscion) and then reinstalling my packages except ImplicitPlots was how I managed to update StaticArrays. In other words, removing the package that was holding things back was not enough. StaticArrays was held back even after removing the package with the outdated manifest. Throwing in Pkg.resolve() didn’t do a thing for me. Also, conservatively removing the line with the old version number and/or the entire StaticArrays entry in the Manifest.toml causes a complete crash (the file header does say "# This file is machine-generated - editing it directly is not advised", so no hard feelings.
you get out of the package REPL mode (like all other REPL modes AFAIK) by pressing backspace.
Oh excellent, that should have been obvious. On my keyboard that’s the “delete” key. help?> takes you out immediately after returning info and you can get out of it with a simple “return”. Meanwhile in vscode/juno, the “escape” key does that kind of thing. So I never got to learn about backspace.
):
