# Is there a way to prevent Julia from updating the registries almost every time?

**URL:** <https://discourse.julialang.org/t/is-there-a-way-to-prevent-julia-from-updating-the-registries-almost-every-time/64150>\
**Category:** Package Management\
**Tags:** git, update\
**Created:** [July 6, 2021, 2:31pm UTC](https://discourse.julialang.org/t/is-there-a-way-to-prevent-julia-from-updating-the-registries-almost-every-time/64150 "2021-07-06T14:31:54Z")\
**Posts on this page:** 20\
**Page:** 2

<div class="post-metadata">

**Author:** ![jdad](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jdad/32/4739_2.png) [@jdad](https://discourse.julialang.org/u/jdad)\
**Post date:** [July 8, 2021, 7:40am UTC](https://discourse.julialang.org/t/is-there-a-way-to-prevent-julia-from-updating-the-registries-almost-every-time/64150/21 "2021-07-08T07:40:29Z")

</div>

Just a further comment about size of registry transfers. You are talking about hundred of megabytes, which hints me that it is a Git tar that is downloaded (with 1.7, the tar is only 3MB). If you are on 1.6, you could try to change registry type by `registry rm General / registry add General` which should change the transfer to tar of data only, if I am not mistaken.

My 2cents, hoping I am not too wrong …

---

<div class="post-metadata">

**Author:** ![GunnarFarneback](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gunnarfarneback/32/1827_2.png) [@GunnarFarneback](https://discourse.julialang.org/u/GunnarFarneback)\
**Post date:** [July 8, 2021, 8:00am UTC](https://discourse.julialang.org/t/is-there-a-way-to-prevent-julia-from-updating-the-registries-almost-every-time/64150/22 "2021-07-08T08:00:35Z")

</div>

> [@dlakelan](#):
>
> The registry filesystem tree is quite large and this means a lot of time to traverse it. I realize this is a bit of an edge case.

Not really an edge case since systems where antivirus software slowly checks every file update has similar characteristics. That is why Julia 1.7 introduces and defaults to the option of not unpacking the registry at all. I recommend that you try the Julia 1.7 beta to see if it works better for you. You should make sure to remove your unpacked registry though, e.g. `pkg> registry rm General`.

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [July 8, 2021, 8:10am UTC](https://discourse.julialang.org/t/is-there-a-way-to-prevent-julia-from-updating-the-registries-almost-every-time/64150/23 "2021-07-08T08:10:53Z")

</div>

> [@tchr](#):
>
> Aside from the improvements coming in 1.7 for Windows users, it still seems to be a valid concern that so much network traffic is generated just to force registry updates (I didn’t know it was on the order of 100 MB, at least).

Where does this 100 MB number come from? It should be about 3.5 MB / update.

---

<div class="post-metadata">

**Author:** ![bernhard](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bernhard/32/2619_2.png) [@bernhard](https://discourse.julialang.org/u/bernhard)\
**Post date:** [July 8, 2021, 8:31am UTC](https://discourse.julialang.org/t/is-there-a-way-to-prevent-julia-from-updating-the-registries-almost-every-time/64150/24 "2021-07-08T08:31:03Z")

</div>

for reference on Windows 10 and a farily new notebook.

EDIT: this is on Julia **1.5.4**

```julia
julia> @time Pkg.Registry.update()
   Updating registry at `C:\Users\bernhard.konig\.julia\registries\General`
   Updating git-repo `https://github.com/JuliaRegistries/General.git`
  8.320919 seconds (1.89 M allocations: 102.974 MiB, 1.16% gc time)

```

---

<div class="post-metadata">

**Author:** ![StefanKarpinski](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefankarpinski/32/24_2.png) [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)\
**Post date:** [July 8, 2021, 8:36am UTC](https://discourse.julialang.org/t/is-there-a-way-to-prevent-julia-from-updating-the-registries-almost-every-time/64150/25 "2021-07-08T08:36:10Z")

</div>

That’s the amount of memory allocated by Julia via GC during the timed operation, not the size of anything transferred. Git updates transfer less data than tarball updates because git only sends the changes since the last update (with history, so it’s a bit larger than just sending a patch, but still). The negotiation can take a while but it should not be transferring hundreds of megabytes of data.

---

<div class="post-metadata">

**Author:** ![HanD](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/hand/32/213908_2.png) [@HanD](https://discourse.julialang.org/u/HanD)\
**Post date:** [July 8, 2021, 9:01am UTC](https://discourse.julialang.org/t/is-there-a-way-to-prevent-julia-from-updating-the-registries-almost-every-time/64150/26 "2021-07-08T09:01:58Z")

</div>

> [@kristoffer.carlsson](#):
>
> Where does this 100 MB number come from? It should be about 3.5 MB / update.

Simple: I checked the network data usage on my mobile phone (network is shared via wifi hotspot) before and after the registry update. And it increased by roughly 100 MB. It is good to know that it shouldn’t take that much.

---

<div class="post-metadata">

**Author:** ![HanD](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/hand/32/213908_2.png) [@HanD](https://discourse.julialang.org/u/HanD)\
**Post date:** [July 8, 2021, 9:04am UTC](https://discourse.julialang.org/t/is-there-a-way-to-prevent-julia-from-updating-the-registries-almost-every-time/64150/27 "2021-07-08T09:04:37Z")

</div>

> [@jdad](#):
>
> Just a further comment about size of registry transfers. You are talking about hundred of megabytes, which hints me that it is a Git tar that is downloaded (with 1.7, the tar is only 3MB). If you are on 1.6, you could try to change registry type by `registry rm General / registry add General` which should change the transfer to tar of data only, if I am not mistaken.

Thanks, @jdad, My registry copy was inherited from Julia v1.0, I wasn’t aware that this could cause an issue. I tried what you suggested, and I will check the network usage for future registry updates.

---

<div class="post-metadata">

**Author:** ![giordano](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/giordano/32/2166_2.png) [@giordano](https://discourse.julialang.org/u/giordano)\
**Post date:** [July 8, 2021, 9:05am UTC](https://discourse.julialang.org/t/is-there-a-way-to-prevent-julia-from-updating-the-registries-almost-every-time/64150/28 "2021-07-08T09:05:26Z")

</div>

Do you have the registry as a Git repository? 100+ MB is the size of the entire cloned repository

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [July 8, 2021, 9:06am UTC](https://discourse.julialang.org/t/is-there-a-way-to-prevent-julia-from-updating-the-registries-almost-every-time/64150/29 "2021-07-08T09:06:11Z")

</div>

Even so, a registry update should only do `git fetch` and not reclone the whole repo…

---

<div class="post-metadata">

**Author:** ![HanD](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/hand/32/213908_2.png) [@HanD](https://discourse.julialang.org/u/HanD)\
**Post date:** [July 8, 2021, 9:09am UTC](https://discourse.julialang.org/t/is-there-a-way-to-prevent-julia-from-updating-the-registries-almost-every-time/64150/30 "2021-07-08T09:09:31Z")

</div>

> [@giordano](#):
>
> Do you have the registry as a Git repository? 100+ MB is the size of the entire cloned repository

> [@kristoffer.carlsson](#):
>
> Even so, a registry update should only do `git fetch` and not reclone the whole repo…

Yes, I have an unpacked Git repo, and I’m also surprised why the update used so much traffic. I agree it shouldn’t. I’m hoping that it was just a local quirk, and the solution suggested by @jdad, i.e., remove and readd the General registry should solve that.

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [July 8, 2021, 9:11am UTC](https://discourse.julialang.org/t/is-there-a-way-to-prevent-julia-from-updating-the-registries-almost-every-time/64150/31 "2021-07-08T09:11:45Z")

</div>

> [@HanD](#):
>
> remove and readd the General registry should solve that.

I don’t think that should do anything.

- If you are using the PkgServer, it is not doing incremental updates anyway so it will download that full 3.5MB file all the time.
- If you are using git, it will just need to reclone the whole registry before it goes back to incrementally fetching.

My guess is something was off with your measurement.

---

<div class="post-metadata">

**Author:** ![StefanKarpinski](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefankarpinski/32/24_2.png) [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)\
**Post date:** [July 8, 2021, 10:06am UTC](https://discourse.julialang.org/t/is-there-a-way-to-prevent-julia-from-updating-the-registries-almost-every-time/64150/32 "2021-07-08T10:06:28Z")

</div>

If the last update was a very long time in the past then the incremental update might have been large. Frequent incremental updates should not be that large.

---

<div class="post-metadata">

**Author:** ![dlakelan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlakelan/32/8491_2.png) [@dlakelan](https://discourse.julialang.org/u/dlakelan)\
**Post date:** [July 8, 2021, 11:24am UTC](https://discourse.julialang.org/t/is-there-a-way-to-prevent-julia-from-updating-the-registries-almost-every-time/64150/33 "2021-07-08T11:24:49Z")

</div>

> [@GunnarFarneback](#):
>
> `pkg> registry rm General` .

Haha, last time I did this it was an hour or more to delete. I will grab 1.7 beta!

---

<div class="post-metadata">

**Author:** ![StefanKarpinski](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefankarpinski/32/24_2.png) [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)\
**Post date:** [July 8, 2021, 11:59am UTC](https://discourse.julialang.org/t/is-there-a-way-to-prevent-julia-from-updating-the-registries-almost-every-time/64150/34 "2021-07-08T11:59:20Z")

</div>

Yikes. The things they call file systems these days.

---

<div class="post-metadata">

**Author:** ![bernhard](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bernhard/32/2619_2.png) [@bernhard](https://discourse.julialang.org/u/bernhard)\
**Post date:** [July 8, 2021, 12:59pm UTC](https://discourse.julialang.org/t/is-there-a-way-to-prevent-julia-from-updating-the-registries-almost-every-time/64150/35 "2021-07-08T12:59:07Z")

</div>

> [@bernhard](#):
>
> `8.320919`

My post was mainly about the time (about 8 seconds compared to frederik’s 0.04 seconds, albeit on a different Julia version). I think my last fetch was only a few days ago.  
Personally I don’t see why I should care at all about the data transferred.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [July 8, 2021, 1:39pm UTC](https://discourse.julialang.org/t/is-there-a-way-to-prevent-julia-from-updating-the-registries-almost-every-time/64150/36 "2021-07-08T13:39:08Z")

</div>

> [@StefanKarpinski](#):
>
> Yikes. The things they call file systems these days.

Offtopic: is there a good technical explanation somewhere online for why Windows filesystem operations are so slow? I don’t understand why they’ve tolerated this being so bad for so long.

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [July 8, 2021, 1:48pm UTC](https://discourse.julialang.org/t/is-there-a-way-to-prevent-julia-from-updating-the-registries-almost-every-time/64150/37 "2021-07-08T13:48:30Z")

</div>

I think the basic answer is that ntfs is 25 years old. Mac got hfs+ in 2005 and apfs in 2015(ish) Linux got ext3, ext4, and now zfs. Windows just never got a good file system (probably for backwards compatibility reasons). Open zfs runs on windows though, so I have some hope they will over time switch to it, but that’s fairly optimistic on my part.

---

<div class="post-metadata">

**Author:** ![fredrikekre](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/fredrikekre/32/1688_2.png) [@fredrikekre](https://discourse.julialang.org/u/fredrikekre)\
**Post date:** [July 8, 2021, 1:56pm UTC](https://discourse.julialang.org/t/is-there-a-way-to-prevent-julia-from-updating-the-registries-almost-every-time/64150/38 "2021-07-08T13:56:22Z")

</div>

See also [Pkg 1.5.2: clone git registries by default on Windows · Issue #2014 · JuliaLang/Pkg.jl · GitHub](https://github.com/JuliaLang/Pkg.jl/issues/2014#issuecomment-690936584) for slow tar unpacking on Windows.

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [July 8, 2021, 2:02pm UTC](https://discourse.julialang.org/t/is-there-a-way-to-prevent-julia-from-updating-the-registries-almost-every-time/64150/39 "2021-07-08T14:02:02Z")

</div>

> [@stevengj](#):
>
> Offtopic: is there a good technical explanation somewhere online for why Windows filesystem operations are so slow? I don’t understand why they’ve tolerated this being so bad for so long.

For the registry, we hit one of the points mentioned in: [Gregory Szorc's Digital Home | Surprisingly Slow](https://gregoryszorc.com/blog/2021/04/06/surprisingly-slow/)

> ### Closing File Handles on Windows
> 
> …  
> While I didn’t realize it at the time, the cause for this was/is Windows Defender. Windows Defender (and other anti-virus / scanning software) typically work on Windows by installing what’s called a filesystem filter driver. This is a kernel driver that essentially hooks itself into the kernel and receives callbacks on I/O and filesystem events. It turns out the close file callback triggers scanning of written data. And this scanning appears to occur synchronously, blocking CloseHandle() from returning. This adds milliseconds of overhead. The net effect is that file mutation I/O on Windows is drastically reduced by Windows Defender and other A/V scanners.  
> As far as I can tell, as long as Windows Defender (and presumably other A/V scanners) are running, there’s no way to make the Windows I/O APIs consistently fast.

---

<div class="post-metadata">

**Author:** ![PetrKryslUCSD](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/petrkryslucsd/32/215825_2.png) [@PetrKryslUCSD](https://discourse.julialang.org/u/PetrKryslUCSD)\
**Post date:** [July 8, 2021, 2:10pm UTC](https://discourse.julialang.org/t/is-there-a-way-to-prevent-julia-from-updating-the-registries-almost-every-time/64150/40 "2021-07-08T14:10:32Z")

</div>

I presume (hope) this also applies to 1.8?

[Previous page](https://discourse.julialang.org/t/is-there-a-way-to-prevent-julia-from-updating-the-registries-almost-every-time/64150.md?page=1)

[Next page](https://discourse.julialang.org/t/is-there-a-way-to-prevent-julia-from-updating-the-registries-almost-every-time/64150.md?page=3)