Feature request: add a reinstall_all_packages to Pkg

General Usage
1

I suggest adding a simple function (`reinstall_all_packages``?) to Pkg that would

  • purge all of .julia, except startup.jl and Project.toml (EDIT: plus some dev and Pluto stuff that I forgot about), and
  • do an update

Motivation: it seems as if I have to do this now and then (more often lately) and I believe this might be daunting for new users. Hence, suggest to add this help function.

2 Likes
2

This seems like it should be investigated further. What problems do you have that this solves?

3

I haven’t had to do this myself in some time, but if .julia ends up in a corrupt state during Pkg operations, it’s painful to describe exactly what needs to be nuked to new users (make sure you don’t touch dev/ or pluto_notebooks/ and probably environments/, but artifacts/, compiled/ and packages/ can go). Yes, Pkg should do things gracefully such that a corrupt state is never attained, but it would still be nice to have a safe-ish reset button for cases of misbehavior.

4

Doesn’t even have to be Pkg misbehaving, could be filesystem corruption for example.

5

Symptoms: packages that suddenly fail to build or precompile, with error messages referring to artifacts and caches. Sometimes Conda. Yesterday, a few packages was updated and suddenly a third of all packages (eg. StatsBase) refused to precompile and the error messages pointed towards ChainRulesCore’s cache.

Solution: nothing that the usual tricks (uninstall, wipe the package specific folder, setting state variables like for GR) solves.

6

Hm, that’s not great but purging all of .julia is not really the right solution I think. If julia chokes on corrupt precompile files it should just delete those and regenerate a new one. If you could provide concrete examples with what the error output is I think that could help understand what the concrete issue is.

7

Thanks,. But the purging worked well and was quick. I think many people resort to that now and then, which was the reason for the feature request.
Anyhow, this is how one error message started (and then there were more when I tried to call on some packages):

Plots → UnitfulExt
  51 dependencies successfully precompiled in 431 seconds. 228 already precompiled.
  1 dependency failed but may be precompilable after restarting julia
  5 dependencies had warnings during precompilation:
┌ SpecialFunctionsChainRulesCoreExt [9eb7bdd4-e44c-55fc-b9cc-1a32cb715188]
│  ┌ Warning: Module ChainRulesCore with build ID fafbfcfd-80b7-b370-0003-63404fd73df5 is missing from the cache.
│  │ This may mean ChainRulesCore [d360d2e6-b24c-11e9-a2a3-2a2ae2dbcce4] does not support precompilation but is imported by a module that does.
│  └ @ Base loading.jl:1793
│  ┌ Error: Error during loading of extension LogExpFunctionsChainRulesCoreExt of LogExpFunctions, use `Base.retry_load_extensions()` to retry.
│  │   exception =
│  │    1-element ExceptionStack:
│  │    Declaring __precompile__(false) is not allowed in files that are being precompiled.
│  │    Stacktrace:
│  │      [1] _require(pkg::Base.PkgId, env::Nothing)
│  │        @ Base .\loading.jl:1825
│  │      [2] _require_prelocked(uuidkey::Base.PkgId, env::Nothing)
│  │        @ Base .\loading.jl:1660
│
8

One known issue is that a virus scanner puts a compilation artifact in quarantaine. Another issue can appear if pre-compilation was interrupted before it was finished. Or it failed due to lack of free memory. A third known issue is that pre-compilation of a package is broken for a new version of Julia, and that got fixed, but the broken artifacts are still somewhere in the .julia folder.

To have an equivalent to

make clean
make

in Julia would be nice.

1 Like
9

Adding something like GitHub - KristofferC/PkgFsck.jl: *Teleports Behind You* Nothing Personal, Kid to the package manager (to fix pkg> fsck · Issue #1717 · JuliaLang/Pkg.jl · GitHub) would indeed be a good idea. That’s for checking and fixing corrupted artifacts and packages.

2 Likes