If I manually remove a package directory from the repository I cannot normally install it. Example:
Step 1. I manually removed package directory from ~/.julia/packages/DataFrames/
Step 2. I try to install it using add DataFrames. I get an error:
(v1.0) pkg> add DataFrames
Resolving package versions...
ERROR: path ~/.julia/packages/DataFrames/KUWId for package DataFrames no longer exists. Remove the package or `develop` it at a new path
Step 3. I remove the package (v1.0) pkg> rm DataFrames - all seems fine.
Step 4. I try to add the package (v1.0) pkg> add DataFrames and get the same error
Step 5. I try to add the package master: (v1.0) pkg> add DataFrames#master - now it works
Would it be possible to make (v1.0) pkg> add DataFrames work?
Yes - it seems that add DataFrames expects the folder to exist if it was manually removed, but e.g. add DataFrames#master creates the folder so it works.
Thank you for investigating into it. Here are detailed instructions how you can reproduce it.
Current version of DataFrames 0.15.2 installs in ~/.julia/packages/DataFrames/KUWId.
Run add DataFrames#master
Delete ~/.julia/packages/DataFrames/KUWId folder
Run add DataFrames (without master) - you get the error
Run rm DataFrames - seems to work
Run add DataFrames (without master) - you get the same error still
Run add DataFrames#master - now it is OK
And a second way (more problematic in my opinion):
Run add DataFrames#master
Run rm DataFrames - removes the package from the registry
Delete ~/.julia/packages/DataFrames/KUWId folder - it should be OK (the package is not installed now)
Run add DataFrames - you get an error, although you try to “freshly” install the package
Now create ~/.julia/packages/DataFrames/KUWId folder (it is empty)
Run add DataFrames - Julia tells you that DataFrames v0.15.2+ #master is installed (although I did not ask for master) and what is more ~/.julia/packages/DataFrames/KUWIdis still empty (although last command went through without a problem informing me that the installation was successfull)
Additionally I have the following question I stumbled when writing down the steps to reproduce the above:
Now the package is corrupded, so I try running add DataFrames to fix the corrupted state, but actually the folder ~/.julia/packages/DataFrames/KUWId does not get fixed
FWIW, I observed a similar issue when ] up SomePackage. However, somewhat strangely it always worked the second time (exact same command, i.e. ] up SomePackage).
In general it seems to me that the Package Manager probably caches something and then uses this information again when processing the commands.
Probably this is needed to ensure responsiveness (and this is great in comparison to an old package manager). But maybe:
there are some checks that are cheap and could be done
you could have a switch that forces the Package Manager to be in a “defensive” mode and check everything when running the command
Well, the files are write-protected, so you have to go to some length to actually remove files like this. But perhaps checking for the entry file makes sense, but since we can’t know if any other files have been removed I am not surehow much it helps to check for one file.
On Windows they are not write-protected (and I must admit with shame that sometimes I dabble with the repo if I have to make a quick-hack and I am to lazy to run develop).
A strategy could be e.g. to have a fingerprint of the state of the repository and checking if it has not changed (and if it changed then we know that the folder is corrupted). For most packages this should be quick (but if it happens to be slow then as I have proposed above there should be some option for turning on strict checking on demand).
Anyway - thank you for https://github.com/JuliaLang/Pkg.jl/issues/913 as hopefully this will solve most of the problems. What I am writing above is nice-to-have only, as this is normally not strictly required.
(and I must admit with shame that sometimes I dabble with the repo if I have to make a quick-hack and I am to lazy to run