# Why does Pkg.activate(".") require compilation? Time to first project activation

**URL:** <https://discourse.julialang.org/t/why-does-pkg-activate-require-compilation-time-to-first-project-activation/101193>\
**Category:** Performance\
**Tags:** pkg, ttfx\
**Created:** [July 5, 2023, 2:34am UTC](https://discourse.julialang.org/t/why-does-pkg-activate-require-compilation-time-to-first-project-activation/101193 "2023-07-05T02:34:32Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![mkitti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkitti/32/12459_2.png) [@mkitti](https://discourse.julialang.org/u/mkitti)\
**Post date:** [July 5, 2023, 2:34am UTC](https://discourse.julialang.org/t/why-does-pkg-activate-require-compilation-time-to-first-project-activation/101193/1 "2023-07-05T02:34:33Z")

</div>

`Pkg.activate(".")` seems to to take about 200 milliseconds, mostly due to compilation. It still takes 40 milliseconds even after activating the same environment again.

```julia
$ julia -e 'using Pkg; for i in 1:5; @time Pkg.activate("."); end'
  Activating project at `~/.julia/dev/ScriptUtils`
  0.220025 seconds (232.01 k allocations: 16.980 MiB, 98.52% compilation time)
  Activating project at `~/.julia/dev/ScriptUtils`
  0.042125 seconds (81.89 k allocations: 5.737 MiB, 95.17% compilation time)
  Activating project at `~/.julia/dev/ScriptUtils`
  0.001953 seconds (3.73 k allocations: 414.523 KiB)
  Activating project at `~/.julia/dev/ScriptUtils`
  0.001633 seconds (3.73 k allocations: 414.180 KiB)
  Activating project at `~/.julia/dev/ScriptUtils`
  0.001591 seconds (3.73 k allocations: 414.086 KiB)

```

I thought perhaps the second round of compilation had to due with the environment changing. Even if I initialize Julia with the correct environment these compilation times persist.

```julia
$ julia --project=. -e 'using Pkg; for i in 1:5; @time Pkg.activate("."); end'
  Activating project at `~/.julia/dev/ScriptUtils`
  0.219066 seconds (231.95 k allocations: 16.975 MiB, 98.48% compilation time)
  Activating project at `~/.julia/dev/ScriptUtils`
  0.051085 seconds (81.89 k allocations: 5.737 MiB, 95.46% compilation time)
  Activating project at `~/.julia/dev/ScriptUtils`
  0.001949 seconds (3.73 k allocations: 414.492 KiB)
  Activating project at `~/.julia/dev/ScriptUtils`
  0.001591 seconds (3.73 k allocations: 414.148 KiB)
  Activating project at `~/.julia/dev/ScriptUtils`
  0.001618 seconds (3.73 k allocations: 414.055 KiB)

```

If I want to simplify usage of Julia code from the command line via an executable script, a `Pkg.activate` call is usually one of the first tasks.

Are we not precompiling `Pkg.activate(::String)` into the system image?

---

<div class="post-metadata">

**Author:** ![mkitti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkitti/32/12459_2.png) [@mkitti](https://discourse.julialang.org/u/mkitti)\
**Post date:** [July 5, 2023, 2:45am UTC](https://discourse.julialang.org/t/why-does-pkg-activate-require-compilation-time-to-first-project-activation/101193/2 "2023-07-05T02:45:12Z")

</div>

It appears we are indeed not precompiling `Pkg.activate` at all.

> <https://github.com/JuliaLang/Pkg.jl/blob/2cd30ad9203e1b331b84387580304dd86492e936/src/Pkg.jl#L867-L885>

Edit: There is actually one precompilation of `Pkg.activate`, but it is for an empty Project.toml.

> <https://github.com/JuliaLang/Pkg.jl/blob/2cd30ad9203e1b331b84387580304dd86492e936/src/Pkg.jl#L800-L801>

Activating an empty Project.toml is fast, but perhaps this is not a useful case.

```julia
$ julia -e 'using Pkg; touch("Project.toml"); @time Pkg.activate(".")'
  Activating project at `~/src/Pkg.jl/test/blah`
  0.000665 seconds (525 allocations: 51.023 KiB)

```

---

<div class="post-metadata">

**Author:** ![mkitti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkitti/32/12459_2.png) [@mkitti](https://discourse.julialang.org/u/mkitti)\
**Post date:** [July 5, 2023, 3:00am UTC](https://discourse.julialang.org/t/why-does-pkg-activate-require-compilation-time-to-first-project-activation/101193/3 "2023-07-05T03:00:18Z")

</div>

I submitted a pull request.

> <https://github.com/JuliaLang/Pkg.jl/pull/3540>
>
> \`Pkg.activate\` is a necessary first step for running executable Julia scripts.
> F…or a novice Julia user, it is often easier to provide them with a script to run simply via \`julia script.jl\`.
> 
> script.jl may contain the following contents.
> \`\`\`
> \#!/bin/env julia
> using Pkg
> Pkg.activate("/path/to/the/right/environment")
> using PackageInEnvironment
> ...
> \`\`\`
> 
> While trying to build such scripts, I found that \`Pkg.activate\` is contributing some latency. 
> 
> Activating the environment at https://github.com/mkitti/ScriptUtils.jl/tree/main/examples/Echo.jl
> can take 100 to 200 milliseconds on my computer, with most of the time being compilation. Even a
> second activation requires an additional 40 mlliseconds of compilation time.
> 
> \`\`\`
> $ julia --project=. -e 'using Pkg; for i in 1:5; @time Pkg.activate("."); end'
> Activating project at \`~/.julia/dev/ScriptUtils/examples/Echo.jl\`
> 0.106829 seconds (159.75 k allocations: 11.833 MiB, 97.05% compilation time)
> Activating project at \`~/.julia/dev/ScriptUtils/examples/Echo.jl\`
> 0.042188 seconds (81.92 k allocations: 5.750 MiB, 95.15% compilation time)
> Activating project at \`~/.julia/dev/ScriptUtils/examples/Echo.jl\`
> 0.001958 seconds (3.76 k allocations: 419.430 KiB)
> Activating project at \`~/.julia/dev/ScriptUtils/examples/Echo.jl\`
> 0.001606 seconds (3.76 k allocations: 419.180 KiB)
> Activating project at \`~/.julia/dev/ScriptUtils/examples/Echo.jl\`
> 0.001637 seconds (3.76 k allocations: 419.133 KiB)
> \`\`\`
> 
> Alternating between the default environment and the current directory, also seems to involve compilation
> up to four activations.
> 
> \`\`\`
> $ julia --project=. -e 'using Pkg; for i in 1:5; @time Pkg.activate("."); @time Pkg.activate(); end'
> Activating project at \`~/src/Pkg.jl\`
> 0.115365 seconds (73.03 k allocations: 5.318 MiB, 99.10% compilation time)
> Activating project at \`/tmp/tmp.298dOvl2Nq/environments/v1.9\`
> 0.124273 seconds (166.67 k allocations: 12.133 MiB, 97.72% compilation time)
> Activating project at \`~/src/Pkg.jl\`
> 0.004956 seconds (742 allocations: 76.609 KiB, 88.74% compilation time)
> Activating project at \`/tmp/tmp.298dOvl2Nq/environments/v1.9\`
> 0.038794 seconds (81.91 k allocations: 5.739 MiB, 94.59% compilation time)
> Activating project at \`~/src/Pkg.jl\`
> 0.000546 seconds (727 allocations: 75.516 KiB)
> Activating project at \`/tmp/tmp.298dOvl2Nq/environments/v1.9\`
> 0.001853 seconds (3.77 k allocations: 417.180 KiB)
> Activating project at \`~/src/Pkg.jl\`
> 0.000377 seconds (728 allocations: 75.812 KiB)
> Activating project at \`/tmp/tmp.298dOvl2Nq/environments/v1.9\`
> 0.001656 seconds (3.77 k allocations: 417.273 KiB)
> Activating project at \`~/src/Pkg.jl\`
> 0.000365 seconds (727 allocations: 75.516 KiB)
> Activating project at \`/tmp/tmp.298dOvl2Nq/environments/v1.9\`
> 0.001650 seconds (3.77 k allocations: 417.180 KiB)
> \`\`\`
> 
> As such I propose adding four \`Pkg.activate\` calls to the Pkg.jl precompilation script, alternating
> between the default environment and TestPkg.jl.

Edit: Doing this with a local build seems to have helped with the default environment, but there is still some compilation needed to activate one of my test enviornments.

```julia
$ ./julia -e 'using Pkg; for i in 1:5; @time Pkg.activate(); end' --trace-compile=stderr;
  Activating project at `/tmp/tmp.298dOvl2Nq/environments/v1.11`
  0.000518 seconds (543 allocations: 41.523 KiB)
  Activating project at `/tmp/tmp.298dOvl2Nq/environments/v1.11`
  0.000251 seconds (538 allocations: 41.133 KiB)
  Activating project at `/tmp/tmp.298dOvl2Nq/environments/v1.11`
  0.000215 seconds (538 allocations: 41.133 KiB)
  Activating project at `/tmp/tmp.298dOvl2Nq/environments/v1.11`
  0.000213 seconds (538 allocations: 41.133 KiB)
  Activating project at `/tmp/tmp.298dOvl2Nq/environments/v1.11`
  0.000205 seconds (539 allocations: 41.430 KiB)

$ ./julia -e 'using Pkg; for i in 1:5; @time Pkg.activate("/home/mkitti/.julia/dev/ScriptUtils/examples/Echo.jl"); end' --trace-compile=stderr;
  Activating project at `~/.julia/dev/ScriptUtils/examples/Echo.jl`
precompile(Tuple{Type{Base.Generator{I, F} where F where I}, Pkg.Types.var"#52#55"{String, String}, Array{Any, 1}})
precompile(Tuple{typeof(Base.collect_similar), Array{Any, 1}, Base.Generator{Array{Any, 1}, Pkg.Types.var"#52#55"{String, String}}})
precompile(Tuple{Pkg.Types.var"#52#55"{String, String}, Base.Dict{String, Any}})
precompile(Tuple{Type{Array{Dates.DateTime, 1}}, UndefInitializer, Tuple{Int64}})
precompile(Tuple{typeof(Base.collect_to_with_first!), Array{Dates.DateTime, 1}, Dates.DateTime, Base.Generator{Array{Any, 1}, Pkg.Types.var"#52#55"{String, String}}, Int64})
precompile(Tuple{typeof(Core.Compiler.eltype), Type{Array{UInt64, 1}}})
precompile(Tuple{typeof(Base.deepcopy_internal), Tuple{UInt64}, Base.IdDict{Any, Any}})
precompile(Tuple{typeof(Base.Ryu.writefixed), Float64, Int64})
  0.110661 seconds (134.34 k allocations: 10.412 MiB, 96.54% compilation time)
  Activating project at `~/.julia/dev/ScriptUtils/examples/Echo.jl`
precompile(Tuple{typeof(Base.ident_cmp), Tuple{UInt64}, Tuple{UInt64}})
  0.009558 seconds (9.16 k allocations: 830.914 KiB, 77.94% compilation time)
  Activating project at `~/.julia/dev/ScriptUtils/examples/Echo.jl`
  0.001870 seconds (3.44 k allocations: 406.727 KiB)
  Activating project at `~/.julia/dev/ScriptUtils/examples/Echo.jl`
  0.001649 seconds (3.44 k allocations: 406.727 KiB)
  Activating project at `~/.julia/dev/ScriptUtils/examples/Echo.jl`
  0.001662 seconds (3.44 k allocations: 407.023 KiB)

```