Very slow "Updating registry at 'C:\Users\me\.julia\registries\General'"

I’m using win10 and julia-1.5.3 from the REPL. When I use Pkg mode to add a new package (or other operations) it starts by updating the registries\General:

(@v1.5) pkg> add Pluto
Updating registry at C:\Users\me\.julia\registries\General

where the updating line can take multiple minutes…every…time…

This seems to be a regression from Pkg adds from a year ago.

  • Is there a debug option to see what is taking so long?
  • Has anyone else seen this slow down?
  • Is there a fix/solution to the problem?

NOTE: This can be very problematic when I load a Pluto notebook and it starts by adding the needed packages to the session!

2 Likes

You can do the following:

pkg> registry rm General

julia> ENV["JULIA_PKG_SERVER"] = ""

pkg> registry add General

Now it will update using Git which should be faster on windows. Only need to do this once.

7 Likes

Looks promising so far! The full clone of General.git process took less time than the update via the package server. I assume git updates will be even faster going forward. Thanks!

Hello Kristoffer
I have a similar Problem. I use also win10 and Julia-1.5.3. It worked perfect during years. Since about 10 days, I can’t load Packages from the General registry e.g. Plots and others (see next)

(v1.5) pkg> add Plots
Updating registry at d:\Users\Heinz\.julia\registries\JuliaComputingRegistry
ERROR: The following package names could not be resolved:

  • Plots (not found in project, manifest or registry)

(v1.5) pkg> st --project
Status D:\Users\Heinz\.julia\environments\v1.5\Project.toml
[44cfe95a] Pkg
[9a3f8284] Random

(v1.5) pkg> st --manifest
Status D:\Users\Heinz\.julia\environments\v1.5\Manifest.toml
[2a0f44e3] Base64
[ade2ca70] Dates
[b77e0a4c] InteractiveUtils
[76f85450] LibGit2
[8f399da3] Libdl
[56ddb016] Logging
[d6f4376e] Markdown
[44cfe95a] Pkg
[de0858da] Printf
[3fa0cd96] REPL
[9a3f8284] Random
[ea8e919c] SHA
[9e88b42a] Serialization
[6462fe0b] Sockets
[cf7118a7] UUIDs
[4ec0a83e] Unicode

(v1.5) pkg>

(v1.5) pkg> registry st
Registry Status
[bbcd6645] JuliaComputingRegistry (https://github.com/JuliaComputing/JuliaComputingRegistry)

As you suggested, I removede the registry and then added it again. Then

(v1.5) pkg> registry add General
Cloning registry from “https://github.com/JuliaRegistries/General.git
ERROR: failed to clone from https://github.com/JuliaRegistries/General.git, error: GitError(Code:ERROR, Class:OS, failed to send request: Der Servername oder die Serveradresse konnte nicht verarbeitet werden.

Pleas can you help me?
Thanks!

Are you using JuliaPro or standard Julia?

Hi,
I’m using standard Juli. After having had this problem, I tried also Julia Pro.But there I had the same kind of errors.
I checkt “registry”, but with the following error messages, I coud’t get further.

(v1.5) pkg> registry up
Updating registry at d:\Users\Heinz\.julia\registries\JuliaComputingRegistry

(v1.5) pkg> add Plots
ERROR: The following package names could not be resolved:

  • Plots (not found in project, manifest or registry)

Try clear out d:\Users\Heinz\.julia\registries completely.

Other than that, are you behind some proxy or got any special network settings that might interfere with the registry download?

Thanks a lot, I will try this tomorrow.
To your question concerning special network settings: Can the LOAD_PATH setting have some influence on the observed behavior and how must this setting be? How can the LOAD_PATH be changed?

This does indeed seem to work. I’m curious though, prior to registry rm General this was my registry:

pkg> registry st
Registry Status
 [23338594] General (https://github.com/JuliaRegistries/General.git) 

is this not already Git?

In fact, this is the exact same output I get after making the change. Why is the update time all of a sudden so different?

Good morning Kristoffer
Thank you again for your hint. It worked!
After deleting “registry” and deinstalling julia 1.5.3, I reinstalled Julia 1.5.3.
After starting Julia and “registry status” there was no registry. Then I added registry with “registry add General” and it was done!
The reason for the problems are still unknown. But lets now move on!
Thank you again for your help.

PS. A good side-effect was, that I learned a lot more about Pkg.jl and the Registry!

Good morning Kristoffer
Thank you again for your hint. It worked!
After deleting “registry” and deinstalling julia 1.5.3, I reinstalled Julia 1.5.3.
After starting Julia and “registry status” there was no registry. Then I added registry with “registry add General” and it was done!
The reason for the problems are still unknown. But lets now move on!
Thank you again for your help.

PS. A good side-effect was, that I learned a lot more about Pkg.jl and the Registry!

It should be, I am not sure why it didn’t work.

Hmm…

As a side note, when I added a package to my default environment today the registry update again took well over a minute :man_shrugging:

I suppose this is not the end of the world, but a bit frustrating that simply communicating with the registry comes with so much time overhead.

Perhaps this is off-topic, but I found the Windows experience frustrating enough that I installed Julia to WSL instead. So far this is a significant improvement over ‘ordinary Windows’, shell mode is nicer, too.

I wonder if there are any drawbacks, or is this basically the best solution for windows?

1 Like

It’s bad on Windows. Should be better in the upcoming 1.6. For now you can do:

julia> ENV["JULIA_PKG_SERVER"] = ""

pkg> registry rm General

pkg> registry add General

and you will get a git version of the registry instead which should be faster to update.

1 Like

Depending on what is required, WSL is probably a good solution. I did try it and the improvements in speed were very good.

However I also run a Linux installation in VirtualBox, and it turns out that VirtualBox and WSL can’t be installed at the same time. I had to choose between the two, so I chose VirtualBox.

Both Linux’y systems were much faster than Windows (all running on the same hardware, of course). Without any real knowledge on my part, my guess is that a lot of this is because the Windows file system is really slow.

This is what I already did…

Thanks for the tip, but unfortunately I’m doing some of this setup for work and trying to convince our IT that we should have WSL installed is just not a battle I’m willing to wage at the moment :sweat_smile: