Pkg.add("FourierFlows") takes long time -- did I do something wrong?

I have committed the first tagged version for FourierFlows.jl #12685 and it was merged successfully.

However, when I do Pkg.add("FourierFlows") on a brand new clean installation I get

julia> Pkg.init()
INFO: Initializing package repository /Users/navid/.julia/v0.6
INFO: Cloning METADATA from

julia> @time Pkg.add("FourierFlows")
INFO: Cloning cache of BinDeps from
INFO: Cloning cache of BufferedStreams from
INFO: Cloning cache of Compat from
INFO: Cloning cache of DataStructures from
INFO: Cloning cache of FileIO from
INFO: Cloning cache of FourierFlows from

Then it stays at this point for ~40 min before it proceeds with

INFO: Cloning cache of JLD2 from
INFO: Cloning cache of Libz from
INFO: Cloning cache of SpecialFunctions from
INFO: Installing BufferedStreams v0.3.3
INFO: Installing DataStructures v0.7.4
INFO: Installing FileIO v0.6.1
INFO: Installing FourierFlows v0.0.1
INFO: Installing JLD2 v0.0.5
INFO: Installing Libz v0.2.4
INFO: Installing SpecialFunctions v0.3.7
INFO: Building SpecialFunctions
INFO: Package database updated
INFO: METADATA is out-of-date — you may not have the latest version of FourierFlows
INFO: Use `Pkg.update()` to get the latest versions of your packages
2756.170447 seconds (5.68 M allocations: 388.634 MiB, 0.01% gc time)

Isn’t this strange? Why Pkg.add() needs 40 min? Did I do something wrong when I registered the package?

For the record:

julia> versioninfo()
Julia Version 0.6.2
Commit d386e40c17 (2017-12-13 18:08 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin17.3.0)
  CPU: Intel(R) Core(TM) i7-4980HQ CPU @ 2.80GHz
  LAPACK: libopenblas64_
  LIBM: libopenlibm
  LLVM: libLLVM-3.9.1 (ORCJIT, haswell)

By the way on a linux machine Pkg.add("FourierFlows") takes ~20sec – much more reasonable.

1 Like

Let me add some additional information that might be relevant.

Initially the repository was in my personal github account: navidcy
After I submitted the PR in METADATA.jl for registering v0.0.1 I created an organization FourierFlows and moved the repository there. I have updated the repositories url in METADATA.jl (PR #12762).

Is this, by any chance, responsible for the long time Pkg.add() is taking?

No, that wouldn’t do anything.

Are you installing from a different computer? Behind a proxy?

I’m not behind a proxy.
I’ve tried from various internet connections and also @glwagner tried from his own – same problem. Hopefully when we submit the v0.0.2 release this issue will be resolved. But any suggestions are welcome!

Maybe it’s related to having __precompile__() on every page? I’ve never seen that before so I actually have no idea what that would do. A single __precompile__() is all that’s necessary, and at least something to try first.

Yes, I’ve already fixed that in the latest version of FourierFlows – thanks.

For the record: I can reproduce the problem in my mac.

This is installing FourierFlows on Pkg3:

It would be interesting if you could try it with Pkg3 ( on the computer where the extreme slowness is ocurring.

@kristoffer.carlsson : Indeed. Pkg3 installed FourierFlows in ~20sec.

So still this problem comes up erratically…
Pkg.add("FourierFlows") gets stuck at:

julia> Pkg.add("FourierFlows")
INFO: Cloning cache of FourierFlows from

Mind blowing. I never had this problem with any other packages other than FourierFlows.

Just tried it here on 0.6.2. It’s been a good 20 minutes and it’s still not done. The julia process is still receiving bytes though; 480 MB downloaded so far. Could it be that FourierFlows has large objects in its git history? Pkg3 doesn’t download the whole git history, so that’s a big difference.

Edit: 55 minutes total on my system :slight_smile:.

I hadn’t read this earlier. Can reproduce this too; about 40 seconds on my machine (though on a different network as well). But still, on Linux, df -sh .git reports 848 MB, which seems excessive.

Could be as well (although I don’t know how Pkg2 add is implemented). Note that that issue is closed as a duplicate of an issue that is still open.

@tkoolen: So, have I made something wrong in the repository? Perhaps at some point in its history there was an accidental push of large files. Is there a way to clean it?

The standard LibGit2 clone is fast:

julia> @time LibGit2.clone("", "FourierFlows")
  3.818350 seconds (466.33 k allocations: 27.295 MiB, 0.12% gc time)

What does Pkg.add use?

Another piece of the puzzle: with the standard LibGit2 clone or when cloning from the command line, the .git directory ends up being ~1.7 MB, not 848 MB as with Pkg.add.

1 Like

@tkoolen: Indeed that was the problem, the repository was clogged up from some accidental commit of many .png files. Now everything is sorted out.