# Why is the update of the registries so slow?

**URL:** <https://discourse.julialang.org/t/why-is-the-update-of-the-registries-so-slow/28659>\
**Category:** General Usage\
**Created:** [September 11, 2019, 6:24pm UTC](https://discourse.julialang.org/t/why-is-the-update-of-the-registries-so-slow/28659 "2019-09-11T18:24:32Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![PetrKryslUCSD](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/petrkryslucsd/32/215825_2.png) [@PetrKryslUCSD](https://discourse.julialang.org/u/PetrKryslUCSD)\
**Post date:** [September 11, 2019, 6:24pm UTC](https://discourse.julialang.org/t/why-is-the-update-of-the-registries-so-slow/28659/1 "2019-09-11T18:24:32Z")

</div>

```julia
$ julia
               _
   _ _ _(_)_ | Documentation: https://docs.julialang.org
  (_) | (_) (_) |
   _ _ _| |_ __ _ | Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` | |
  | | |_| | | | (_| | | Version 1.2.0 (2019-08-20)
 _/ |\ __'_|_|_|\__'_| | Official https://julialang.org/ release
|__/ |

julia> t = time(); include("go.jl"); println("$(time()-t)")
Activating environment at `C:\Users\PK\Documents\work\ACol\Project.toml`
  Updating registry at `C:\Users\PK\.julia\registries\General`
  Updating git-repo `https://github.com/JuliaRegistries/General.git`
45.980000019073486

```

Fourty six seconds. Why?

---

<div class="post-metadata">

**Author:** ![kevbonham](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kevbonham/32/216165_2.png) [@kevbonham](https://discourse.julialang.org/u/kevbonham)\
**Post date:** [September 11, 2019, 6:46pm UTC](https://discourse.julialang.org/t/why-is-the-update-of-the-registries-so-slow/28659/2 "2019-09-11T18:46:36Z")

</div>

Slow internet connection? Mine usually completes in ~5 sec

---

<div class="post-metadata">

**Author:** ![PetrKryslUCSD](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/petrkryslucsd/32/215825_2.png) [@PetrKryslUCSD](https://discourse.julialang.org/u/PetrKryslUCSD)\
**Post date:** [September 11, 2019, 6:52pm UTC](https://discourse.julialang.org/t/why-is-the-update-of-the-registries-so-slow/28659/3 "2019-09-11T18:52:11Z")

</div>

This is on 10 Mpbs. You?

---

<div class="post-metadata">

**Author:** ![chakravala](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chakravala/32/6832_2.png) [@chakravala](https://discourse.julialang.org/u/chakravala)\
**Post date:** [September 11, 2019, 6:54pm UTC](https://discourse.julialang.org/t/why-is-the-update-of-the-registries-so-slow/28659/4 "2019-09-11T18:54:39Z")

</div>

There is definitely a minute long lag with the package manager every now and then for me. This is especially true if I use the package manager after first loading or precompiling a larger package. That’s why it seems to me that sometimes it is better to open a new instance of Julia to get a quicker response from Pkg.

Also, I’m on Linux not Windows.

---

<div class="post-metadata">

**Author:** ![fredrikekre](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/fredrikekre/32/1688_2.png) [@fredrikekre](https://discourse.julialang.org/u/fredrikekre)\
**Post date:** [September 11, 2019, 7:14pm UTC](https://discourse.julialang.org/t/why-is-the-update-of-the-registries-so-slow/28659/5 "2019-09-11T19:14:37Z")

</div>

Looks like you are using Windows: LibGit2 operations are generally pretty slow there from my experience.

---

<div class="post-metadata">

**Author:** ![aaowens](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/aaowens/32/12101_2.png) [@aaowens](https://discourse.julialang.org/u/aaowens)\
**Post date:** [September 11, 2019, 7:22pm UTC](https://discourse.julialang.org/t/why-is-the-update-of-the-registries-so-slow/28659/6 "2019-09-11T19:22:15Z")

</div>

Is it just a Windows thing? I had trouble explaining to my students why installing packages in a fresh Julia install takes over 1 minute on the Windows school computers, but only a few seconds on my Linux laptop. That would explain it.

---

<div class="post-metadata">

**Author:** ![00vareladavid](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/00vareladavid/32/23521_2.png) [@00vareladavid](https://discourse.julialang.org/u/00vareladavid)\
**Post date:** [September 11, 2019, 7:26pm UTC](https://discourse.julialang.org/t/why-is-the-update-of-the-registries-so-slow/28659/7 "2019-09-11T19:26:49Z")

</div>

Yes, `git` is (of course) very snappy on Linux

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [September 11, 2019, 7:38pm UTC](https://discourse.julialang.org/t/why-is-the-update-of-the-registries-so-slow/28659/8 "2019-09-11T19:38:51Z")

</div>

Could people on Windows look at the process monitor while the registry is updating and see if Windows Defender is taking up a lot of CPU during that.

Also, does adding the `.julia/registries/General` directory to the exclusion list of Windows Defender help at all?

---

<div class="post-metadata">

**Author:** ![sdanisch](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sdanisch/32/1406_2.png) [@sdanisch](https://discourse.julialang.org/u/sdanisch)\
**Post date:** [September 11, 2019, 10:07pm UTC](https://discourse.julialang.org/t/why-is-the-update-of-the-registries-so-slow/28659/9 "2019-09-11T22:07:34Z")

</div>

It’s not only a windows issue, I’d say:

 ![image](https://global.discourse-cdn.com/julialang/original/3X/a/6/a69c8f828c0a2faebeccd9868526627270821edd.png)  
The article:

> **[\- Nextjournal](https://nextjournal.com/a/LdUzmn2NrtzFdo63YZThU?token=TycJH4EFJc4K6XvbU4FvFZ)**

Hard to compare package installations between machines, but after Pkg3, I’ve never noticed windows to be particularly slow anymore…

---

<div class="post-metadata">

**Author:** ![00vareladavid](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/00vareladavid/32/23521_2.png) [@00vareladavid](https://discourse.julialang.org/u/00vareladavid)\
**Post date:** [September 11, 2019, 10:50pm UTC](https://discourse.julialang.org/t/why-is-the-update-of-the-registries-so-slow/28659/10 "2019-09-11T22:50:25Z")

</div>

Can you share the toml files for this environment (before the first `up`)?

---

<div class="post-metadata">

**Author:** ![PetrKryslUCSD](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/petrkryslucsd/32/215825_2.png) [@PetrKryslUCSD](https://discourse.julialang.org/u/PetrKryslUCSD)\
**Post date:** [September 12, 2019, 5:21am UTC](https://discourse.julialang.org/t/why-is-the-update-of-the-registries-so-slow/28659/11 "2019-09-12T05:21:53Z")

</div>

Good thinking. During the fetching of the registry the “anti-malware service executable” was over 30% of CPU usage.

I added the folder `.julia/registries` to the exclusion list. That didn’t work.  
I added `.julia` to the exclusion list, and then the fetching of the registries did not result in high anti-malware executable CPU usage. However when the fetching was done, the anti-malware executable went up to over 50% of CPU usage.

---

<div class="post-metadata">

**Author:** ![bernhard](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bernhard/32/2619_2.png) [@bernhard](https://discourse.julialang.org/u/bernhard)\
**Post date:** [September 12, 2019, 5:44am UTC](https://discourse.julialang.org/t/why-is-the-update-of-the-registries-so-slow/28659/12 "2019-09-12T05:44:00Z")

</div>

I have no explicit comment to your question. But I would say Defender, Anitmalware blabla (possibly this is the same thing), and some other windows processes that I do not understand (wmi host provider, etc.) tend to slow Julia (and other software) by a huge margin.  
I actually have the feeling, Microsoft has a deal with energy providers.

---

<div class="post-metadata">

**Author:** ![PetrKryslUCSD](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/petrkryslucsd/32/215825_2.png) [@PetrKryslUCSD](https://discourse.julialang.org/u/PetrKryslUCSD)\
**Post date:** [September 12, 2019, 5:52am UTC](https://discourse.julialang.org/t/why-is-the-update-of-the-registries-so-slow/28659/13 "2019-09-12T05:52:08Z")

</div>

Turning off antivirus software (not necessarily advisable), resulted in a speedup of around 3 (16 seconds compared to 46 seconds originally). I suspect there is an additional speedup that might be gained by turning off indexing (the Windows indexer is running towards the end of the instantiation).

---

<div class="post-metadata">

**Author:** ![lobingera](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lobingera/32/211_2.png) [@lobingera](https://discourse.julialang.org/u/lobingera)\
**Post date:** [September 12, 2019, 9:18am UTC](https://discourse.julialang.org/t/why-is-the-update-of-the-registries-so-slow/28659/14 "2019-09-12T09:18:08Z")

</div>

Long time ago the registry (METADATA) was a wide (~2500 subdirectories) git repository with small individual files; so a database as directory structure. This took very long to update via git pull.

The Pkg3 happened. And everything went to be way-more-efficient.

Now the Registry is a tree (1-level Intial char, 2-nd level Package name) of small individual files, so a database as directory structure. Could it be that this takes long as updates via git pull?

---

<div class="post-metadata">

**Author:** ![Yifan\_Liu](https://avatars.discourse-cdn.com/v4/letter/y/4da419/32.png) [@Yifan\_Liu](https://discourse.julialang.org/u/Yifan_Liu)\
**Post date:** [February 19, 2020, 6:49am UTC](https://discourse.julialang.org/t/why-is-the-update-of-the-registries-so-slow/28659/15 "2020-02-19T06:49:32Z")

</div>

Will this get faster? I still find it is much less responsive than python and r on windows.

---

<div class="post-metadata">

**Author:** ![StefanKarpinski](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefankarpinski/32/24_2.png) [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)\
**Post date:** [February 19, 2020, 7:05am UTC](https://discourse.julialang.org/t/why-is-the-update-of-the-registries-so-slow/28659/16 "2020-02-19T07:05:56Z")

</div>

Yes. You can already use a Julia 1.4 prerelease and set `JULIA_PKG_SERVER` to [geo.pkg.julialang.org](http://geo.pkg.julialang.org) to get registries as tarballs.

---

<div class="post-metadata">

**Author:** ![carstenbauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carstenbauer/32/4981_2.png) [@carstenbauer](https://discourse.julialang.org/u/carstenbauer)\
**Post date:** [February 19, 2020, 7:11am UTC](https://discourse.julialang.org/t/why-is-the-update-of-the-registries-so-slow/28659/17 "2020-02-19T07:11:33Z")

</div>

The link, when clicking on it, redirects me to [eu.pkg.julialang.org](https://eu.pkg.julialang.org) which doesn’t seem to exist.

---

<div class="post-metadata">

**Author:** ![StefanKarpinski](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefankarpinski/32/24_2.png) [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)\
**Post date:** [February 19, 2020, 7:16am UTC](https://discourse.julialang.org/t/why-is-the-update-of-the-registries-so-slow/28659/18 "2020-02-19T07:16:33Z")

</div>

It’s not a website, it serves packages.

---

<div class="post-metadata">

**Author:** ![Yifan\_Liu](https://avatars.discourse-cdn.com/v4/letter/y/4da419/32.png) [@Yifan\_Liu](https://discourse.julialang.org/u/Yifan_Liu)\
**Post date:** [April 30, 2020, 3:12pm UTC](https://discourse.julialang.org/t/why-is-the-update-of-the-registries-so-slow/28659/19 "2020-04-30T15:12:31Z")

</div>

I am using Julia 1.4.0 but the fetching process of packages updating is still very slow.

---

<div class="post-metadata">

**Author:** ![fredrikekre](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/fredrikekre/32/1688_2.png) [@fredrikekre](https://discourse.julialang.org/u/fredrikekre)\
**Post date:** [April 30, 2020, 3:14pm UTC](https://discourse.julialang.org/t/why-is-the-update-of-the-registries-so-slow/28659/20 "2020-04-30T15:14:33Z")

</div>

If you set `$ export JULIA_PKG_SERVER=pkg.julialang.org` (and delete the old git-registry) it should be significantly faster.

Edit: I see this was discussed above, but maybe the problem is that your registry is still in git-form?

[Next page](https://discourse.julialang.org/t/why-is-the-update-of-the-registries-so-slow/28659.md?page=2)