# Pkg.add("FourierFlows") takes long time -- did I do something wrong?

**URL:** <https://discourse.julialang.org/t/pkg-add-fourierflows-takes-long-time-did-i-do-something-wrong/8259>\
**Category:** New to Julia\
**Created:** [January 10, 2018, 12:27am UTC](https://discourse.julialang.org/t/pkg-add-fourierflows-takes-long-time-did-i-do-something-wrong/8259 "2018-01-10T00:27:24Z")\
**Posts on this page:** 18\
**Page:** 1

<div class="post-metadata">

**Author:** ![navidcy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/navidcy/32/2750_2.png) [@navidcy](https://discourse.julialang.org/u/navidcy)\
**Post date:** [January 10, 2018, 12:27am UTC](https://discourse.julialang.org/t/pkg-add-fourierflows-takes-long-time-did-i-do-something-wrong/8259/1 "2018-01-10T00:27:24Z")

</div>

I have committed the first tagged version for FourierFlows.jl #12685 and it was merged successfully.

However, when I do `Pkg.add("FourierFlows")` on a brand new clean installation I get

```julia
julia> Pkg.init()
INFO: Initializing package repository /Users/navid/.julia/v0.6
INFO: Cloning METADATA from https://github.com/JuliaLang/METADATA.jl

julia> @time Pkg.add("FourierFlows")
INFO: Cloning cache of BinDeps from https://github.com/JuliaLang/BinDeps.jl.git
INFO: Cloning cache of BufferedStreams from https://github.com/BioJulia/BufferedStreams.jl.git
INFO: Cloning cache of Compat from https://github.com/JuliaLang/Compat.jl.git
INFO: Cloning cache of DataStructures from https://github.com/JuliaCollections/DataStructures.jl.git
INFO: Cloning cache of FileIO from https://github.com/JuliaIO/FileIO.jl.git
INFO: Cloning cache of FourierFlows from https://github.com/FourierFlows/FourierFlows.jl.git

```

Then it stays at this point for **~40 min** before it proceeds with

```julia
INFO: Cloning cache of JLD2 from https://github.com/simonster/JLD2.jl.git
INFO: Cloning cache of Libz from https://github.com/BioJulia/Libz.jl.git
INFO: Cloning cache of SpecialFunctions from https://github.com/JuliaMath/SpecialFunctions.jl.git
INFO: Installing BufferedStreams v0.3.3
INFO: Installing DataStructures v0.7.4
INFO: Installing FileIO v0.6.1
INFO: Installing FourierFlows v0.0.1
INFO: Installing JLD2 v0.0.5
INFO: Installing Libz v0.2.4
INFO: Installing SpecialFunctions v0.3.7
INFO: Building SpecialFunctions
INFO: Package database updated
INFO: METADATA is out-of-date — you may not have the latest version of FourierFlows
INFO: Use `Pkg.update()` to get the latest versions of your packages
2756.170447 seconds (5.68 M allocations: 388.634 MiB, 0.01% gc time)

```

Isn’t this strange? Why Pkg.add() needs 40 min? Did I do something wrong when I registered the package?

For the record:

```julia
julia> versioninfo()
Julia Version 0.6.2
Commit d386e40c17 (2017-12-13 18:08 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin17.3.0)
  CPU: Intel(R) Core(TM) i7-4980HQ CPU @ 2.80GHz
  WORD_SIZE: 64
  BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Haswell)
  LAPACK: libopenblas64_
  LIBM: libopenlibm
  LLVM: libLLVM-3.9.1 (ORCJIT, haswell)

```

By the way on a linux machine `Pkg.add("FourierFlows")` takes ~20sec – much more reasonable.

---

<div class="post-metadata">

**Author:** ![navidcy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/navidcy/32/2750_2.png) [@navidcy](https://discourse.julialang.org/u/navidcy)\
**Post date:** [January 10, 2018, 7:57pm UTC](https://discourse.julialang.org/t/pkg-add-fourierflows-takes-long-time-did-i-do-something-wrong/8259/2 "2018-01-10T19:57:53Z")

</div>

Let me add some additional information that might be relevant.

Initially the repository was in my personal github account: navidcy  
After I submitted the PR in METADATA.jl for registering v0.0.1 I created an organization FourierFlows and moved the repository there. I have updated the repositories url in METADATA.jl (PR #12762).

Is this, by any chance, responsible for the long time Pkg.add() is taking?

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [January 10, 2018, 8:14pm UTC](https://discourse.julialang.org/t/pkg-add-fourierflows-takes-long-time-did-i-do-something-wrong/8259/3 "2018-01-10T20:14:49Z")

</div>

> [@navidcy](#):
>
> Is this, by any chance, responsible for the long time Pkg.add() is taking?

No, that wouldn’t do anything.

Are you installing from a different computer? Behind a proxy?

---

<div class="post-metadata">

**Author:** ![navidcy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/navidcy/32/2750_2.png) [@navidcy](https://discourse.julialang.org/u/navidcy)\
**Post date:** [January 10, 2018, 8:33pm UTC](https://discourse.julialang.org/t/pkg-add-fourierflows-takes-long-time-did-i-do-something-wrong/8259/4 "2018-01-10T20:33:14Z")

</div>

I’m not behind a proxy.  
I’ve tried from various internet connections and also @glwagner tried from his own – same problem. Hopefully when we submit the v0.0.2 release this issue will be resolved. But any suggestions are welcome!

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [January 10, 2018, 8:40pm UTC](https://discourse.julialang.org/t/pkg-add-fourierflows-takes-long-time-did-i-do-something-wrong/8259/5 "2018-01-10T20:40:23Z")

</div>

Maybe it’s related to having ` __precompile__ ()` on every page? I’ve never seen that before so I actually have no idea what that would do. A single ` __precompile__ ()` is all that’s necessary, and at least something to try first.

---

<div class="post-metadata">

**Author:** ![navidcy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/navidcy/32/2750_2.png) [@navidcy](https://discourse.julialang.org/u/navidcy)\
**Post date:** [January 10, 2018, 8:43pm UTC](https://discourse.julialang.org/t/pkg-add-fourierflows-takes-long-time-did-i-do-something-wrong/8259/6 "2018-01-10T20:43:13Z")

</div>

Yes, I’ve already fixed that in the latest version of FourierFlows – thanks.

---

<div class="post-metadata">

**Author:** ![favba](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/favba/32/2735_2.png) [@favba](https://discourse.julialang.org/u/favba)\
**Post date:** [January 10, 2018, 8:52pm UTC](https://discourse.julialang.org/t/pkg-add-fourierflows-takes-long-time-did-i-do-something-wrong/8259/7 "2018-01-10T20:52:43Z")

</div>

For the record: I can reproduce the problem in my mac.

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [January 11, 2018, 12:50pm UTC](https://discourse.julialang.org/t/pkg-add-fourierflows-takes-long-time-did-i-do-something-wrong/8259/8 "2018-01-11T12:50:07Z")

</div>

This is installing FourierFlows on Pkg3:

[https://giphy.com/gifs/l49JE1ZSFrOgRW5e8/fullscreen](https://giphy.com/gifs/l49JE1ZSFrOgRW5e8/fullscreen)

It would be interesting if you could try it with Pkg3 ([https://github.com/JuliaLang/Pkg3.jl](https://github.com/JuliaLang/Pkg3.jl)) on the computer where the extreme slowness is ocurring.

---

<div class="post-metadata">

**Author:** ![navidcy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/navidcy/32/2750_2.png) [@navidcy](https://discourse.julialang.org/u/navidcy)\
**Post date:** [January 11, 2018, 5:40pm UTC](https://discourse.julialang.org/t/pkg-add-fourierflows-takes-long-time-did-i-do-something-wrong/8259/9 "2018-01-11T17:40:53Z")

</div>

@kristoffer.carlsson : Indeed. Pkg3 installed FourierFlows in ~20sec.

---

<div class="post-metadata">

**Author:** ![navidcy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/navidcy/32/2750_2.png) [@navidcy](https://discourse.julialang.org/u/navidcy)\
**Post date:** [May 5, 2018, 9:55pm UTC](https://discourse.julialang.org/t/pkg-add-fourierflows-takes-long-time-did-i-do-something-wrong/8259/10 "2018-05-05T21:55:58Z")

</div>

So still this problem comes up erratically…  
`Pkg.add("FourierFlows")` gets stuck at:

```julia
julia> Pkg.add("FourierFlows")
INFO: Cloning cache of FourierFlows from https://github.com/FourierFlows/FourierFlows.jl.git

```

Mind blowing. I never had this problem with _any_ other packages other than `FourierFlows`.

---

<div class="post-metadata">

**Author:** ![tkoolen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkoolen/32/1603_2.png) [@tkoolen](https://discourse.julialang.org/u/tkoolen)\
**Post date:** [May 5, 2018, 10:20pm UTC](https://discourse.julialang.org/t/pkg-add-fourierflows-takes-long-time-did-i-do-something-wrong/8259/11 "2018-05-05T22:20:14Z")

</div>

Just tried it here on 0.6.2. It’s been a good 20 minutes and it’s still not done. The `julia` process is still receiving bytes though; 480 MB downloaded so far. Could it be that FourierFlows has large objects in its git history? Pkg3 doesn’t download the whole git history, so that’s a big difference.

Edit: 55 minutes total on my system 🙂.

---

<div class="post-metadata">

**Author:** ![tkoolen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkoolen/32/1603_2.png) [@tkoolen](https://discourse.julialang.org/u/tkoolen)\
**Post date:** [May 5, 2018, 10:43pm UTC](https://discourse.julialang.org/t/pkg-add-fourierflows-takes-long-time-did-i-do-something-wrong/8259/12 "2018-05-05T22:43:26Z")

</div>

> [@navidcy](#):
>
> By the way on a linux machine Pkg.add(“FourierFlows”) takes ~20sec – much more reasonable

I hadn’t read this earlier. Can reproduce this too; about 40 seconds on my machine (though on a different network as well). But still, on Linux, `df -sh .git` reports 848 MB, which seems excessive.

---

<div class="post-metadata">

**Author:** ![tkoolen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkoolen/32/1603_2.png) [@tkoolen](https://discourse.julialang.org/u/tkoolen)\
**Post date:** [May 5, 2018, 11:50pm UTC](https://discourse.julialang.org/t/pkg-add-fourierflows-takes-long-time-did-i-do-something-wrong/8259/13 "2018-05-05T23:50:27Z")

</div>

Could be [https://github.com/libgit2/libgit2/issues/4464](https://github.com/libgit2/libgit2/issues/4464) as well (although I don’t know how Pkg2 `add` is implemented). Note that that issue is closed as a duplicate of an issue that is still open.

---

<div class="post-metadata">

**Author:** ![navidcy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/navidcy/32/2750_2.png) [@navidcy](https://discourse.julialang.org/u/navidcy)\
**Post date:** [May 6, 2018, 10:44am UTC](https://discourse.julialang.org/t/pkg-add-fourierflows-takes-long-time-did-i-do-something-wrong/8259/14 "2018-05-06T10:44:46Z")

</div>

@tkoolen: So, have I made something wrong in the repository? Perhaps at some point in its history there was an accidental push of large files. Is there a way to clean it?

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [May 6, 2018, 10:47am UTC](https://discourse.julialang.org/t/pkg-add-fourierflows-takes-long-time-did-i-do-something-wrong/8259/15 "2018-05-06T10:47:20Z")

</div>

The standard LibGit2 clone is fast:

```julia
julia> @time LibGit2.clone("https://github.com/FourierFlows/FourierFlows.jl", "FourierFlows")
  3.818350 seconds (466.33 k allocations: 27.295 MiB, 0.12% gc time)
LibGit2.GitRepo("/Users/kristoffer/FourierFlows.jl/FourierFlows")

```

---

<div class="post-metadata">

**Author:** ![navidcy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/navidcy/32/2750_2.png) [@navidcy](https://discourse.julialang.org/u/navidcy)\
**Post date:** [May 6, 2018, 11:16am UTC](https://discourse.julialang.org/t/pkg-add-fourierflows-takes-long-time-did-i-do-something-wrong/8259/16 "2018-05-06T11:16:38Z")

</div>

What does `Pkg.add` use?

---

<div class="post-metadata">

**Author:** ![tkoolen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkoolen/32/1603_2.png) [@tkoolen](https://discourse.julialang.org/u/tkoolen)\
**Post date:** [May 6, 2018, 12:36pm UTC](https://discourse.julialang.org/t/pkg-add-fourierflows-takes-long-time-did-i-do-something-wrong/8259/17 "2018-05-06T12:36:27Z")

</div>

Another piece of the puzzle: with the standard LibGit2 clone or when cloning from the command line, the `.git` directory ends up being ~1.7 MB, not 848 MB as with `Pkg.add`.

---

<div class="post-metadata">

**Author:** ![navidcy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/navidcy/32/2750_2.png) [@navidcy](https://discourse.julialang.org/u/navidcy)\
**Post date:** [June 17, 2018, 12:00am UTC](https://discourse.julialang.org/t/pkg-add-fourierflows-takes-long-time-did-i-do-something-wrong/8259/18 "2018-06-17T00:00:37Z")

</div>

@tkoolen: Indeed that was the problem, the repository was clogged up from some accidental commit of many .png files. Now everything is sorted out.