If I manually remove a package directory from the repository I cannot normally install it. Example:
Step 1. I manually removed package directory from
Step 2. I try to install it using
add DataFrames. I get an error:
(v1.0) pkg> add DataFrames
Resolving package versions...
ERROR: path ~/.julia/packages/DataFrames/KUWId for package DataFrames no longer exists. Remove the package or `develop` it at a new path
Step 3. I remove the package
(v1.0) pkg> rm DataFrames - all seems fine.
Step 4. I try to add the package
(v1.0) pkg> add DataFrames and get the same error
Step 5. I try to add the package master:
(v1.0) pkg> add DataFrames#master - now it works
Would it be possible to make
(v1.0) pkg> add DataFrames work?
Maybe Second Pkg.add for branch revision fails · Issue #913 · JuliaLang/Pkg.jl · GitHub (fix incoming with Julia 1.0.3) although that only applied to adding with a branch multiple times. You get this from just a regular
Yes - it seems that
add DataFrames expects the folder to exist if it was manually removed, but e.g.
add DataFrames#master creates the folder so it works.
I can not reproduce this.
Thank you for investigating into it. Here are detailed instructions how you can reproduce it.
Current version of DataFrames 0.15.2 installs in
add DataFrames (without master) - you get the error
rm DataFrames - seems to work
add DataFrames (without master) - you get the same error still
add DataFrames#master - now it is OK
And a second way (more problematic in my opinion):
rm DataFrames - removes the package from the registry
~/.julia/packages/DataFrames/KUWId folder - it should be OK (the package is not installed now)
add DataFrames - you get an error, although you try to “freshly” install the package
- Now create
~/.julia/packages/DataFrames/KUWId folder (it is empty)
add DataFrames - Julia tells you that
DataFrames v0.15.2+ #master is installed (although I did not ask for
master) and what is more
~/.julia/packages/DataFrames/KUWIdis still empty (although last command went through without a problem informing me that the installation was successfull)
Additionally I have the following question I stumbled when writing down the steps to reproduce the above:
- Delete file
- Now the package is corrupded, so I try running
add DataFrames to fix the corrupted state, but actually the folder
~/.julia/packages/DataFrames/KUWId does not get fixed
FWIW, I observed a similar issue when
] up SomePackage. However, somewhat strangely it always worked the second time (exact same command, i.e.
] up SomePackage).
In general it seems to me that the Package Manager probably caches something and then uses this information again when processing the commands.
Probably this is needed to ensure responsiveness (and this is great in comparison to an old package manager). But maybe:
- there are some checks that are cheap and could be done
- you could have a switch that forces the Package Manager to be in a “defensive” mode and check everything when running the command
Well, the files are write-protected, so you have to go to some length to actually remove files like this. But perhaps checking for the entry file makes sense, but since we can’t know if any other files have been removed I am not surehow much it helps to check for one file.
On Windows they are not write-protected (and I must admit with shame that sometimes I dabble with the repo if I have to make a quick-hack and I am to lazy to run
A strategy could be e.g. to have a fingerprint of the state of the repository and checking if it has not changed (and if it changed then we know that the folder is corrupted). For most packages this should be quick (but if it happens to be slow then as I have proposed above there should be some option for turning on strict checking on demand).
Anyway - thank you for https://github.com/JuliaLang/Pkg.jl/issues/913 as hopefully this will solve most of the problems. What I am writing above is nice-to-have only, as this is normally not strictly required.
There is actually another problem here. The problems described above is only partially fixed by #913.
Hmm, seems like a bug then, the
chmod calls here https://github.com/JuliaLang/Pkg.jl/blob/2075b74621f7829a930a480fe4c3b8c8f8ac0924/src/Operations.jl#L65-L74 should set the file permission. Does that not happend? (This requires Julia 1.0.2 btw).
I am on Julia 1.0.2 but I have admin privilleges on my laptop.