# It takes several minutes to add a single package

**URL:** <https://discourse.julialang.org/t/it-takes-several-minutes-to-add-a-single-package/70704>\
**Category:** New to Julia\
**Tags:** pkg, environment\
**Created:** [October 31, 2021, 8:59pm UTC](https://discourse.julialang.org/t/it-takes-several-minutes-to-add-a-single-package/70704 "2021-10-31T20:59:07Z")\
**Posts on this page:** 20\
**Page:** 3

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [November 2, 2021, 1:27pm UTC](https://discourse.julialang.org/t/it-takes-several-minutes-to-add-a-single-package/70704/41 "2021-11-02T13:27:22Z")

</div>

> [@giordano](#):
>
> How’d the time be different if you had run the same command in the global environment?

As bad. The problem is the download of the registries. I will try the `Pkg.offline` option (which, by the way, is [not documented here](https://pkgdocs.julialang.org/v1/), apparently).

The `Pkg.offline` options is what really improves my experience here:

```julia
julia> Pkg.offline(true)

julia> @time Pkg.add("PDBTools")
   Resolving package versions...
    Updating `/tmp/jl_iorxwT/Project.toml`
  [e29189f1] + PDBTools v0.12.11
    Updating `/tmp/jl_iorxwT/Manifest.toml`
  [59287772] + Formatting v0.4.2
  [bac558e1] + OrderedCollections v1.4.1
  [e29189f1] + PDBTools v0.12.11
  [d96e819e] + Parameters v0.12.3
  [3a884ed6] + UnPack v1.0.2
  [2a0f44e3] + Base64
  [b77e0a4c] + InteractiveUtils
  [56ddb016] + Logging
  [d6f4376e] + Markdown
  [de0858da] + Printf
  [9a3f8284] + Random
  [9e88b42a] + Serialization
  [8dfed614] + Test
  [4ec0a83e] + Unicode
  2.467132 seconds (3.48 M allocations: 205.214 MiB, 2.91% gc time, 6.54% compilation time)

```

> [@dilumaluthge](#):
>
> What if you do this at the beginning of your Julia session?

It becomes fast:

```julia
julia> import Pkg

julia> Pkg.UPDATED_REGISTRY_THIS_SESSION[] = true
true

julia> @time Pkg.add("PDBTools")
   Resolving package versions...
  No Changes to `~/.julia/environments/v1.6/Project.toml`
  No Changes to `~/.julia/environments/v1.6/Manifest.toml`
  5.910878 seconds (5.07 M allocations: 320.312 MiB, 1.37% gc time, 5.82% compilation time)

(@v1.6) pkg> activate --temp
  Activating new environment at `/tmp/jl_pMQIWm/Project.toml`

julia> @time Pkg.add("PDBTools")
   Resolving package versions...
    Updating `/tmp/jl_pMQIWm/Project.toml`
  [e29189f1] + PDBTools v0.12.11
    Updating `/tmp/jl_pMQIWm/Manifest.toml`
  [59287772] + Formatting v0.4.2
  [bac558e1] + OrderedCollections v1.4.1
  [e29189f1] + PDBTools v0.12.11
  [d96e819e] + Parameters v0.12.3
  [3a884ed6] + UnPack v1.0.2
  [2a0f44e3] + Base64
  [b77e0a4c] + InteractiveUtils
  [56ddb016] + Logging
  [d6f4376e] + Markdown
  [de0858da] + Printf
  [9a3f8284] + Random
  [9e88b42a] + Serialization
  [8dfed614] + Test
  [4ec0a83e] + Unicode
  0.468187 seconds (807.31 k allocations: 67.419 MiB, 6.85% gc time, 52.59% compilation time)

julia> 

```

> [@GunnarFarneback](#):
>
> Julia 1.7 fares when it’s not unpacking the registry.

The problem is on fetching the registry there (downloading the Julia tarball took 3 seconds):

 ![image](https://global.discourse-cdn.com/julialang/original/3X/a/a/aa2d3785634b71715b9c367eca4dd0022826f663.png)

---

<div class="post-metadata">

**Author:** ![giordano](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/giordano/32/2166_2.png) [@giordano](https://discourse.julialang.org/u/giordano)\
**Post date:** [November 2, 2021, 1:28pm UTC](https://discourse.julialang.org/t/it-takes-several-minutes-to-add-a-single-package/70704/42 "2021-11-02T13:28:20Z")

</div>

> [@Lilith](#):
>
> The essential difference is that you only have to add a package once in the global environment, whereas you need to add it in each new temporary environment you create that needs it.

I feel like you are conflating local environments with temporary environments. I suggest using temporary ones when you want to do a quick test with a new package that you may not use anymore. If you want to have something persistent, you create a local environment.

> [@lmiq](#):
>
> As bad.

So, I really don’t understand the point you’re trying to make about putting all packages in the global environment.

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [November 2, 2021, 1:31pm UTC](https://discourse.julialang.org/t/it-takes-several-minutes-to-add-a-single-package/70704/43 "2021-11-02T13:31:38Z")

</div>

> [@giordano](#):
>
> So, I really don’t understand the point you’re trying to make about putting all packages in the global environment.

Because once the package is in the global environment, I don’t need to install them again in the temporary environments to test things. That is why.

If I am developing let say package “A” in the environment of `A`, and I happen to need some other package there for testing something, I have to add it to that environment, or simply `use` it from the global environment.

As you can see, `using` it is “free”. Adding it to the local environment (now I know that I can improve that by using `Pkg.offline(true)` or the updated registry thing) may take several minutes.

---

<div class="post-metadata">

**Author:** ![dilumaluthge](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dilumaluthge/32/29283_2.png) [@dilumaluthge](https://discourse.julialang.org/u/dilumaluthge)\
**Post date:** [November 2, 2021, 1:32pm UTC](https://discourse.julialang.org/t/it-takes-several-minutes-to-add-a-single-package/70704/44 "2021-11-02T13:32:39Z")

</div>

> [@lmiq](#):
>
> The problem is on fetching the registry there (downloading the Julia tarball took 3 seconds):
> 
> ![image](https://global.discourse-cdn.com/julialang/original/3X/a/a/aa2d3785634b71715b9c367eca4dd0022826f663.png)

From that screenshot, it looks like you are using the Git version of the General registry. In order to get the registry speedups in Julia 1.7 that @GunnarFarneback was talking about, you will need to be using the Pkg server version of the General registry.

In order to switch to the Pkg server version of the General registry, run the following commands:

```julia
julia> import Pkg

julia> Pkg.Registry.rm("General")

julia> delete!(ENV, "JULIA_PKG_SERVER")

julia> Pkg.Registry.add("General")

```

---

<div class="post-metadata">

**Author:** ![giordano](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/giordano/32/2166_2.png) [@giordano](https://discourse.julialang.org/u/giordano)\
**Post date:** [November 2, 2021, 1:32pm UTC](https://discourse.julialang.org/t/it-takes-several-minutes-to-add-a-single-package/70704/45 "2021-11-02T13:32:45Z")

</div>

> [@lmiq](#):
>
> Because once the package is in the global environment, I don’t need to install them again in the temporary environments to test things. That is why.

Then create a local environment? As I said above, my point is to not clutter the global environment. Temporary environments are… well… temporary, I do _ **not** _ recommend them to put packages you know you want to use later on

---

<div class="post-metadata">

**Author:** ![Lilith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lilith/32/27492_2.png) [@Lilith](https://discourse.julialang.org/u/Lilith)\
**Post date:** [November 2, 2021, 1:36pm UTC](https://discourse.julialang.org/t/it-takes-several-minutes-to-add-a-single-package/70704/46 "2021-11-02T13:36:21Z")

</div>

> [@giordano](#):
>
> I feel like you are conflating local environments with temporary environments. I suggest using temporary ones when you want to do a quick test with a new package that you may not use anymore. If you want to have something persistent, you create a local environment.

Perhaps I am misunderstanding “proper” use of local environments.

> [@giordano](#):
>
> Then create a local environment?

Let’s say I have 10 projects, that require an overlapping set of packages, say, each project has 3–5 packages it alone uses, 10–20 packages it shares with a couple other projects, and 5–10 packages that it shares with most projects. (Each of these direct requirements also has its own dependency tree.) When I initially create a package I don not know exactly what packages it will need, so I have to add them incrementally with `]add Xxx Yyy` one or two at a time.

Are you recommending I put 5 packages in my global environment, create 10 local environments and install the remaining requisite 18–30 packages individually in each of the ten local environments?

---

<div class="post-metadata">

**Author:** ![Lilith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lilith/32/27492_2.png) [@Lilith](https://discourse.julialang.org/u/Lilith)\
**Post date:** [November 2, 2021, 1:38pm UTC](https://discourse.julialang.org/t/it-takes-several-minutes-to-add-a-single-package/70704/47 "2021-11-02T13:38:38Z")

</div>

> [@dilumaluthge](#):
>
> ```julia
> julia> import Pkg
> 
> julia> Pkg.Registry.rm("General")
> 
> julia> delete!(ENV, "JULIA_PKG_SERVER")
> 
> julia> Pkg.Registry.add("General")
> 
> ```

Thanks! (Does this have side effects folks should be aware of that span Julia sessions?)

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [November 2, 2021, 1:39pm UTC](https://discourse.julialang.org/t/it-takes-several-minutes-to-add-a-single-package/70704/48 "2021-11-02T13:39:32Z")

</div>

> [@giordano](#):
>
> Then create a local environment? As I said above, my point is to not clutter the global environment. Temporary environments are… well… temporary, I do _ **not** _ recommend them to put packages you know you want to use later on

Let us say that I am developing my package `PDBTools`. Then, I am working in the environment `PDBTools pkg>`. I happen to need to draw a plot, but `Plots` is not a dependency of my package at all.

If `Plots` is installed in the global environment, I just need to do `using Plots`. If not, what do I do?

> [@dilumaluthge](#):
>
> In order to switch to the Pkg server version of the General registry, run the following commands:

I am getting this error message, and it does not seem to be working (or is it?). In any case, it is taking a lot of time anyway:

 ![image](https://global.discourse-cdn.com/julialang/original/3X/8/2/82a495e4f052e23027d7f0f2b0d2fee7c479016f.png)

---

<div class="post-metadata">

**Author:** ![giordano](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/giordano/32/2166_2.png) [@giordano](https://discourse.julialang.org/u/giordano)\
**Post date:** [November 2, 2021, 1:41pm UTC](https://discourse.julialang.org/t/it-takes-several-minutes-to-add-a-single-package/70704/49 "2021-11-02T13:41:30Z")

</div>

> [@Lilith](#):
>
> Are you recommending I put 5 packages in my global environment, create 10 local environments and install the remaining requisite 18–30 packages individually in each of the ten local environments?

That could be a good middle ground, yes.

I tend to be more extreme and have only dev packages (`BenchmarkTools`, `Revise`, little else) in the global environment and then for all other tasks I use local environments. Local environments are useful for reproducibility: if I put _all_ packages I need for an application in its local environment I can check in the manifest with Git and give it to others, who can simply instantiate the environment and have everything ready to use. If you rely on packages installed in your own global environment that’s not going to work.

---

<div class="post-metadata">

**Author:** ![Lilith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lilith/32/27492_2.png) [@Lilith](https://discourse.julialang.org/u/Lilith)\
**Post date:** [November 2, 2021, 1:50pm UTC](https://discourse.julialang.org/t/it-takes-several-minutes-to-add-a-single-package/70704/50 "2021-11-02T13:50:09Z")

</div>

> [@giordano](#):
>
> > [@Lilith](#):
> >
> > Are you recommending I put 5 packages in my global environment, create 10 local environments and install the remaining requisite 18–30 packages individually in each of the ten local environments?
> 
> That could be a good middle ground, yes.

So the issue I see with that approach is that it takes some time to install a single package and when we multiply by 10\*(18–30) we get 180–300 package installations. This is a lot of avoidable work for me because I only actually need to have a reproducible environment for two of those ten projects. With the global environment system I can install all 60 packages I have ever used in my global environment and then only go through manual installation in local environments 36–60 times for a total of 96–120 package installations rather than 180–300.

It’d be even nicer if the 36-60 manual instillations to create reproducible environments was automated, leaving me with only 60 in the global environment (3x–5x savings at the UI level)

I explain this alternative in detail above:

> [@It takes several minutes to add a single package](https://discourse.julialang.org/t/it-takes-several-minutes-to-add-a-single-package/70704/26):
>
> Here’s a potential workflow: Remove “@v#.#” from Base.LOAD\_PATH in startup.jl Typically always work in the global environment and add packages with ]add --preserve=all package. Update with ]up when I run into a package bug, want a new feature, or have a few minutes where it is okay to let Julia be unresponsive and am okay with a risk of having to deal with broken updates. When I want to share a minimum working environment, including whenever I do package development, make a new temporary or p…

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [November 2, 2021, 2:04pm UTC](https://discourse.julialang.org/t/it-takes-several-minutes-to-add-a-single-package/70704/51 "2021-11-02T14:04:38Z")

</div>

> [@Lilith](#):
>
> I explain this alternative in detail above:

From the discussion above, an alternative workflow can be:

1. Start a temporary environment.
2. Add every possible package you may wan to use to that temporary environment (i. e. download everything).
3. Add `Pkg.offline(true)` and perhaps `Pkg.UPDATED_REGISTRY_THIS_SESSION[] = true` to `startup.jl`.

Than you can add packages quickly to new environments, temporary or not, all the time, and the main environment has no package whatsoever.

It still does not solve the fact that you cannot `use` those packages without installing them in each environment, but at least the interaction will be swift.

---

<div class="post-metadata">

**Author:** ![Lilith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lilith/32/27492_2.png) [@Lilith](https://discourse.julialang.org/u/Lilith)\
**Post date:** [November 2, 2021, 2:07pm UTC](https://discourse.julialang.org/t/it-takes-several-minutes-to-add-a-single-package/70704/52 "2021-11-02T14:07:23Z")

</div>

> [@giordano](#):
>
> > [@Lilith](#):
> >
> > FWIW, here is a benchmark of adding packages available locally. Takes a 3.4s in my cluttered global environment and 0.13s in a fresh --temp environment.
> 
> Yes, the difference is between a cluttered environment (where the resolver must keep happy more packages together) and a non-cluttered one.

I’ve profiled, with the following results:  
24% (~1s) resolve  
23% (~1s) precompile  
49% (~2s) handle interrupt (I did not trigger any keyboard interrupts during profiling)

Precompile shouldn’t be happening because there are no changes to the manifest, right?  
And I have no idea what handle interrupt is (`@Base/task.jl:411; (::Pkg.API.var"#212#239"{Bool, Vector{Task}, Pkg.API.var"#handle_interrupt#231"{Base.Event, ReentrantLo...`).

In detail:

```julia
Overhead ╎ [+additional indent] Count File:Line; Function
=========================================================
     ╎26790 @Base/client.jl:485; _start()
     ╎ 26790 @Base/client.jl:302; exec_options(opts::Base.JLOptions)
     ╎ 26790 @Base/client.jl:372; run_main_repl(interactive::Bool, quiet::Bool, banner::Bool, history_file::Bool, color_set::Bool)
     ╎ 26790 @Base/essentials.jl:706; invokelatest
     ╎ 26790 @Base/essentials.jl:708; #invokelatest#2
     ╎ 26790 @Base/client.jl:387; (::Base.var"#874#876"{Bool, Bool, Bool})(REPL::Module)
     ╎ ╎ 26790 ...ackage_macos64/build/usr/share/julia/stdlib/v1.6/REPL/src/REPL.jl:305; run_repl(repl::REPL.AbstractREPL, consumer::Any)
     ╎ ╎ 26790 ...ackage_macos64/build/usr/share/julia/stdlib/v1.6/REPL/src/REPL.jl:317; run_repl(repl::REPL.AbstractREPL, consumer::Any; backend_on_current_task::Bool)
     ╎ ╎ 26790 ...ackage_macos64/build/usr/share/julia/stdlib/v1.6/REPL/src/REPL.jl:185; start_repl_backend(backend::REPL.REPLBackend, consumer::Any)
     ╎ ╎ 26790 ...ckage_macos64/build/usr/share/julia/stdlib/v1.6/REPL/src/REPL.jl:200; repl_backend_loop(backend::REPL.REPLBackend)
     ╎ ╎ 26790 ...ckage_macos64/build/usr/share/julia/stdlib/v1.6/REPL/src/REPL.jl:139; eval_user_input(ast::Any, backend::REPL.REPLBackend)
     ╎ ╎ ╎ 26790 @Base/boot.jl:360; eval
     ╎ ╎ ╎ 26790 REPL[37]:2; top-level scope
     ╎ ╎ ╎ 26790 ...acos64/build/usr/share/julia/stdlib/v1.6/Profile/src/Profile.jl:28; macro expansion
     ╎ ╎ ╎ 26789 @Base/timing.jl:210; macro expansion
     ╎ ╎ ╎ 26789 ...ckage_macos64/build/usr/share/julia/stdlib/v1.6/Pkg/src/API.jl:74; add
     ╎ ╎ ╎ ╎ 26789 ...ckage_macos64/build/usr/share/julia/stdlib/v1.6/Pkg/src/API.jl:74; #add#22
     ╎ ╎ ╎ ╎ 26789 ...kage_macos64/build/usr/share/julia/stdlib/v1.6/Pkg/src/API.jl:75; add
     ╎ ╎ ╎ ╎ 26789 ...kage_macos64/build/usr/share/julia/stdlib/v1.6/Pkg/src/API.jl:75; #add#23
     ╎ ╎ ╎ ╎ 26789 ...kage_macos64/build/usr/share/julia/stdlib/v1.6/Pkg/src/API.jl:77; add(pkgs::Vector{Pkg.Types.PackageSpec})
     ╎ ╎ ╎ ╎ 14423 ...age_macos64/build/usr/share/julia/stdlib/v1.6/Pkg/src/API.jl:79; add(pkgs::Vector{Pkg.Types.PackageSpec}; io::Base.TTY, kwargs::Base.Iterators.Pairs{Union{},...
     ╎ ╎ ╎ ╎ ╎ 14423 ...ge_macos64/build/usr/share/julia/stdlib/v1.6/Pkg/src/API.jl:154; (::Pkg.API.var"#add##kw")(::NamedTuple{(:io,), Tuple{Base.TTY}}, ::typeof(Pkg.API.add), ctx...
     ╎ ╎ ╎ ╎ ╎ 14416 ...ge_macos64/build/usr/share/julia/stdlib/v1.6/Pkg/src/API.jl:203; add(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec}; preserve::Pkg.Types.Prese...
     ╎ ╎ ╎ ╎ ╎ 14416 ...64/build/usr/share/julia/stdlib/v1.6/Pkg/src/Operations.jl:1223; add##kw
     ╎ ╎ ╎ ╎ ╎ 12373 ...64/build/usr/share/julia/stdlib/v1.6/Pkg/src/Operations.jl:1231; add(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec}, new_git::Vector{Base.UU...
     ╎ ╎ ╎ ╎ ╎ 12366 ...4/build/usr/share/julia/stdlib/v1.6/Pkg/src/Operations.jl:1216; _resolve
     ╎ ╎ ╎ ╎ ╎ ╎ 12366 ...4/build/usr/share/julia/stdlib/v1.6/Pkg/src/Operations.jl:1182; tiered_resolve(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec})
     ╎ ╎ ╎ ╎ ╎ ╎ 11831 ...4/build/usr/share/julia/stdlib/v1.6/Pkg/src/Operations.jl:1210; targeted_resolve(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec}, preserve:...
     ╎ ╎ ╎ ╎ ╎ ╎ 9618 ...4/build/usr/share/julia/stdlib/v1.6/Pkg/src/Operations.jl:404; resolve_versions!(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec})
     ╎ ╎ ╎ ╎ ╎ ╎ 4254 .../build/usr/share/julia/stdlib/v1.6/Pkg/src/Operations.jl:534; deps_graph(ctx::Pkg.Types.Context, uuid_to_name::Dict{Base.UUID, String}, reqs::Dict{B...
     ╎ ╎ ╎ ╎ 11944 ...age_macos64/build/usr/share/julia/stdlib/v1.6/Pkg/src/API.jl:80; add(pkgs::Vector{Pkg.Types.PackageSpec}; io::Base.TTY, kwargs::Base.Iterators.Pairs{Union{},...
     ╎ ╎ ╎ ╎ ╎ 11943 ...ge_macos64/build/usr/share/julia/stdlib/v1.6/Pkg/src/Pkg.jl:592; _auto_precompile(ctx::Pkg.Types.Context)
    1╎ ╎ ╎ ╎ ╎ 11943 ...ge_macos64/build/usr/share/julia/stdlib/v1.6/Pkg/src/API.jl:920; (::Pkg.API.var"#precompile##kw")(::NamedTuple{(:internal_call,), Tuple{Bool}}, ::typeof(Pk...
     ╎ ╎ ╎ ╎ ╎ 3737 ...e_macos64/build/usr/share/julia/stdlib/v1.6/Pkg/src/API.jl:921; precompile(ctx::Pkg.Types.Context; internal_call::Bool, strict::Bool, kwargs::Base.Iterat...
     ╎ ╎ ╎ ╎ ╎ 3737 ...e_macos64/build/usr/share/julia/stdlib/v1.6/Pkg/src/API.jl:1304; (::Pkg.API.var"#instantiate##kw")(::NamedTuple{(:allow_autoprecomp,), Tuple{Bool}}, ::ty...
     ╎ ╎ ╎ ╎ ╎ 3669 ..._macos64/build/usr/share/julia/stdlib/v1.6/Pkg/src/API.jl:1335; instantiate(ctx::Pkg.Types.Context; manifest::Nothing, update_registry::Bool, verbose::B...
     ╎ ╎ ╎ ╎ ╎ ╎ 3651 ...4/build/usr/share/julia/stdlib/v1.6/Pkg/src/Operations.jl:131; is_instantiated(ctx::Pkg.Types.Context)
     ╎ ╎ ╎ ╎ ╎ ╎ 3651 @Base/reducedim.jl:886; all
     ╎ ╎ ╎ ╎ ╎ ╎ 3651 @Base/reducedim.jl:886; #all#698
     ╎ ╎ ╎ ╎ ╎ ╎ 3651 @Base/reduce.jl:923; _all
     ╎ ╎ ╎ ╎ ╎ ╎ 3651 .../build/usr/share/julia/stdlib/v1.6/Pkg/src/Operations.jl:131; #13
     ╎ ╎ ╎ ╎ ╎ ╎ ╎ 3589 ...build/usr/share/julia/stdlib/v1.6/Pkg/src/Operations.jl:1786; is_package_downloaded(ctx::Pkg.Types.Context, pkg::Pkg.Types.PackageSpec)
     ╎ ╎ ╎ ╎ ╎ ╎ ╎ 3589 ...build/usr/share/julia/stdlib/v1.6/Pkg/src/Operations.jl:713; check_artifacts_downloaded
    5╎ ╎ ╎ ╎ ╎ 4157 ...e_macos64/build/usr/share/julia/stdlib/v1.6/Pkg/src/API.jl:981; precompile(ctx::Pkg.Types.Context; internal_call::Bool, strict::Bool, kwargs::Base.Iterat...
     ╎ ╎ ╎ ╎ ╎ 4143 ...e_macos64/build/usr/share/julia/stdlib/v1.6/Pkg/src/API.jl:978; (::Pkg.API.var"#in_deps#227")(pkgs::Vector{Base.PkgId}, deps::Vector{Base.PkgId}, dmap::D...
     ╎ ╎ ╎ ╎ ╎ 4143 @Base/reducedim.jl:883; any
     ╎ ╎ ╎ ╎ ╎ ╎ 4143 @Base/reducedim.jl:883; #any#696
     ╎ ╎ ╎ ╎ ╎ ╎ 4143 @Base/reduce.jl:876; _any(f::Pkg.API.var"#204#228"{Vector{Base.PkgId}, Dict{Base.PkgId, Vector{Base.PkgId}}}...
     ╎ ╎ ╎ ╎ ╎ ╎ 4143 ..._macos64/build/usr/share/julia/stdlib/v1.6/Pkg/src/API.jl:978; (::Pkg.API.var"#204#228"{Vector{Base.PkgId}, Dict{Base.PkgId, Vector{Base.PkgId}}})(dep...
     ╎ ╎ ╎ ╎ ╎ ╎ 4061 ...macos64/build/usr/share/julia/stdlib/v1.6/Pkg/src/API.jl:978; (::Pkg.API.var"#in_deps#227")(pkgs::Vector{Base.PkgId}, deps::Vector{Base.PkgId}, dma...
     ╎ ╎ ╎ ╎ ╎ ╎ 4061 @Base/reducedim.jl:883; any
     ╎ ╎ ╎ ╎ ╎ ╎ ╎ 4061 @Base/reducedim.jl:883; #any#696
     ╎ ╎ ╎ ╎ ╎ ╎ ╎ 4057 @Base/reduce.jl:876; _any(f::Pkg.API.var"#204#228"{Vector{Base.PkgId}, Dict{Base.PkgId, Vector{Base.PkgId...
     ╎ ╎ ╎ ╎ ╎ ╎ ╎ 4057 ...acos64/build/usr/share/julia/stdlib/v1.6/Pkg/src/API.jl:978; (::Pkg.API.var"#204#228"{Vector{Base.PkgId}, Dict{Base.PkgId, Vector{Base.PkgId}}})(...
     ╎ ╎ ╎ ╎ ╎ ╎ ╎ 3798 ...cos64/build/usr/share/julia/stdlib/v1.6/Pkg/src/API.jl:978; (::Pkg.API.var"#in_deps#227")(pkgs::Vector{Base.PkgId}, deps::Vector{Base.PkgId}, ...
     ╎ ╎ ╎ ╎ ╎ ╎ ╎ 3798 @Base/reducedim.jl:883; any
     ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ 3798 @Base/reducedim.jl:883; #any#696
     ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ 3789 @Base/reduce.jl:876; _any(f::Pkg.API.var"#204#228"{Vector{Base.PkgId}, Dict{Base.PkgId, Vector{Base.Pk...
   12╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ 3789 ...os64/build/usr/share/julia/stdlib/v1.6/Pkg/src/API.jl:978; (::Pkg.API.var"#204#228"{Vector{Base.PkgId}, Dict{Base.PkgId, Vector{Base.PkgId}}...
     ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ 3474 ...s64/build/usr/share/julia/stdlib/v1.6/Pkg/src/API.jl:978; (::Pkg.API.var"#in_deps#227")(pkgs::Vector{Base.PkgId}, deps::Vector{Base.PkgId...
     ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ 3474 @Base/reducedim.jl:883; any
     ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ 3474 @Base/reducedim.jl:883; #any#696
     ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ 3469 @Base/reduce.jl:876; _any(f::Pkg.API.var"#204#228"{Vector{Base.PkgId}, Dict{Base.PkgId, Vector{Base...
    9╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ 3469 ...64/build/usr/share/julia/stdlib/v1.6/Pkg/src/API.jl:978; (::Pkg.API.var"#204#228"{Vector{Base.PkgId}, Dict{Base.PkgId, Vector{Base.PkgI...
    1╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ 3286 ...4/build/usr/share/julia/stdlib/v1.6/Pkg/src/API.jl:978; (::Pkg.API.var"#in_deps#227")(pkgs::Vector{Base.PkgId}, deps::Vector{Base.Pk...
     ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ 3285 @Base/reducedim.jl:883; any
     ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ 3285 @Base/reducedim.jl:883; #any#696
     ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ 3280 @Base/reduce.jl:876; _any(f::Pkg.API.var"#204#228"{Vector{Base.PkgId}, Dict{Base.PkgId, Vector{B...
   11╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ 3280 .../build/usr/share/julia/stdlib/v1.6/Pkg/src/API.jl:978; (::Pkg.API.var"#204#228"{Vector{Base.PkgId}, Dict{Base.PkgId, Vector{Base.P...
     ╎ ╎ ╎ ╎ ╎ 3409 ...e_macos64/build/usr/share/julia/stdlib/v1.6/Pkg/src/API.jl:1098; precompile(ctx::Pkg.Types.Context; internal_call::Bool, strict::Bool, kwargs::Base.Iterat...
     ╎ ╎ ╎ ╎ ╎ 3138 @Base/loading.jl:290; locate_package(pkg::Base.PkgId)
   11╎25689 @Base/task.jl:411; (::Pkg.API.var"#212#239"{Bool, Vector{Task}, Pkg.API.var"#handle_interrupt#231"{Base.Event, ReentrantLo...
     ╎ 24550 ...er/package_macos64/build/usr/share/julia/stdlib/v1.6/Pkg/src/API.jl:1126; macro expansion
    1╎ 24542 ...er/package_macos64/build/usr/share/julia/stdlib/v1.6/Pkg/src/API.jl:904; _is_stale
    6╎ 8281 @Base/loading.jl:1716; stale_cachefile(modpath::String, cachefile::String)
     ╎ 6712 @Base/loading.jl:290; locate_package(pkg::Base.PkgId)
    4╎ 5416 @Base/loading.jl:396; manifest_uuid_path(env::String, pkg::Base.PkgId)
    3╎ 14504 @Base/loading.jl:1750; stale_cachefile(modpath::String, cachefile::String)
     ╎ 13681 @Base/loading.jl:254; identify_package(where::Base.PkgId, name::String)
   65╎ 12903 @Base/loading.jl:379; manifest_deps_get(env::String, where::Base.PkgId, name::String)
   45╎ ╎ 3048 @Base/loading.jl:503; explicit_manifest_deps_get(project_file::String, where::Base.UUID, name::String)
     ╎ ╎ 3003 @Base/uuid.jl:82; UUID

```

---

<div class="post-metadata">

**Author:** ![Lilith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lilith/32/27492_2.png) [@Lilith](https://discourse.julialang.org/u/Lilith)\
**Post date:** [November 2, 2021, 2:09pm UTC](https://discourse.julialang.org/t/it-takes-several-minutes-to-add-a-single-package/70704/53 "2021-11-02T14:09:30Z")

</div>

> [@lmiq](#):
>
> Start a temporary environment.

This should probably be a local environment so that you can keep it across Julia sessions.

> [@lmiq](#):
>
> Than you can add packages quickly to new environments

That depends on the environment, I’ve experienced between .1 and 8 seconds depending on the environment and the time of day (all without using any internet connection).

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [November 2, 2021, 2:21pm UTC](https://discourse.julialang.org/t/it-takes-several-minutes-to-add-a-single-package/70704/54 "2021-11-02T14:21:19Z")

</div>

> [@giordano](#):
>
> if I put _all_ packages I need for an application in its local environment I can check in the manifest with Git and give it to others, who can simply instantiate the environment and have everything ready to use

I completely understand the importance and utility of this, but this is _not_ how people use packages, nor never will be. No package user guide starts saying to the user to download a Manifest file and instantiate anything. Manuals say: install the package, start using it.

I think most people is coming in contact with environments not because of the features, but because they are a sort of annoying workaround for the fact that just using a lot of packages in the global environment starts to become problematic whenever a new package or update occurs.

I my humble opinion, making Julia deal well with a cluttered main environment would be a significant user experience improvement for the majority of users, and requires a more conservative approach to updates (update only if necessary, warn the user about the version available, etc.).

---

<div class="post-metadata">

**Author:** ![dlakelan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlakelan/32/8491_2.png) [@dlakelan](https://discourse.julialang.org/u/dlakelan)\
**Post date:** [November 2, 2021, 2:22pm UTC](https://discourse.julialang.org/t/it-takes-several-minutes-to-add-a-single-package/70704/55 "2021-11-02T14:22:01Z")

</div>

> [@lmiq](#):
>
> If what I experience in the cluster was my everyday experience, I would never think of using Julia for anything.

Does your cluster use glusterfs for storage? Also did you try Julia 1.7? A lot of work was put into speeding up package management on windows by not crawling directories. This also helps enormously for glusterfs and maybe also for other NFS based filesystems. I think you need to install 1.7 and rm the existing registry then add it back.

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [November 2, 2021, 2:23pm UTC](https://discourse.julialang.org/t/it-takes-several-minutes-to-add-a-single-package/70704/56 "2021-11-02T14:23:21Z")

</div>

> [@dlakelan](#):
>
> Does your cluster use glusterfs for storage? Also did you try Julia 1.7?

No, and yes (it seems to be something related to the access to git repositories, I don’t know). It is not _my_ cluster. It is the cluster I have access to.

---

<div class="post-metadata">

**Author:** ![giordano](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/giordano/32/2166_2.png) [@giordano](https://discourse.julialang.org/u/giordano)\
**Post date:** [November 2, 2021, 2:28pm UTC](https://discourse.julialang.org/t/it-takes-several-minutes-to-add-a-single-package/70704/57 "2021-11-02T14:28:59Z")

</div>

> [@lmiq](#):
>
> I completely understand the importance and utility of this, but this is _not_ how people use packages, nor never will be. No package user guide starts saying to the user to download a Manifest file and instantiate anything. Manuals say: install the package, start using it.

I’m talking about applications I want to share with other people, or maybe reproduce myself on other systems, or even on the same machine later in time. I don’t think I said that’s what should be written in packages user guides?

> [@lmiq](#):
>
> I my humble opinion, making Julia deal well with a cluttered main environment would be a significant user experience improvement for the majority of users, and requires a more conservative approach to updates (update only if necessary, warn the user about the version available, etc.).

It’s a matter of fact packages do have compat bounds. The more packages you have in a single environment, the more constraints the resolver needs to satisfy, the more likely you’ll be to have some packages held back by some weird combinations. You’re happy with that?

---

<div class="post-metadata">

**Author:** ![Lilith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lilith/32/27492_2.png) [@Lilith](https://discourse.julialang.org/u/Lilith)\
**Post date:** [November 2, 2021, 2:30pm UTC](https://discourse.julialang.org/t/it-takes-several-minutes-to-add-a-single-package/70704/58 "2021-11-02T14:30:07Z")

</div>

> [@lmiq](#):
>
> From the discussion above, an alternative workflow can be…

> [@Lilith](#):
>
> Here’s a potential workflow…

Yet another:

1. Create an environment called `everything` or `workspace` or `all`
2. Move all packages in `global` to `all`
3. Activate `all` in `startup.jl`
4. Work in `all` by default, adding new packages as you come across them
5. When necessary, create a local environment for reproducibility.

This also benefits from manual configuration of `]add` (e.g. not automatically updating everything)

---

<div class="post-metadata">

**Author:** ![Lilith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lilith/32/27492_2.png) [@Lilith](https://discourse.julialang.org/u/Lilith)\
**Post date:** [November 2, 2021, 2:33pm UTC](https://discourse.julialang.org/t/it-takes-several-minutes-to-add-a-single-package/70704/59 "2021-11-02T14:33:08Z")

</div>

> [@giordano](#):
>
> The more packages you have in a single environment, the more constraints the resolver needs to satisfy, the more likely you’ll be to have some packages held back by some weird combinations. You’re happy with that?

I actually don’t think these fundamental limitations are the issue here. In my use cases, I do not have problematic compat bounds in the packages I’m using, and the majority of my inconvenience comes from unnecessary pre-compilation, whatever makes `]add` slow other than resolve, and the need to manually futz with environments.

---

<div class="post-metadata">

**Author:** ![dlakelan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlakelan/32/8491_2.png) [@dlakelan](https://discourse.julialang.org/u/dlakelan)\
**Post date:** [November 2, 2021, 2:34pm UTC](https://discourse.julialang.org/t/it-takes-several-minutes-to-add-a-single-package/70704/60 "2021-11-02T14:34:23Z")

</div>

> [@lmiq](#):
>
> No, and yes (it seems to be something related to the access to git repositories, I don’t know).

Doing the `] reg rm General` then `]reg add General` should dramatically speed things in 1.7 it won’t use git at all. This only affects your account not other users of the cluster.

I’ve got my home dir on a glusterfs directory and for me the registry updates were taking tens of minutes to an hour. Installing 1.7 and doing that dropped it to tens of seconds

[Previous page](https://discourse.julialang.org/t/it-takes-several-minutes-to-add-a-single-package/70704.md?page=2)

[Next page](https://discourse.julialang.org/t/it-takes-several-minutes-to-add-a-single-package/70704.md?page=4)