# How to measure precompilation time?

**URL:** <https://discourse.julialang.org/t/how-to-measure-precompilation-time/102617>\
**Category:** Profiling\
**Tags:** precompilation\
**Created:** [August 8, 2023, 9:43pm UTC](https://discourse.julialang.org/t/how-to-measure-precompilation-time/102617 "2023-08-08T21:43:03Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/27854_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [August 8, 2023, 9:43pm UTC](https://discourse.julialang.org/t/how-to-measure-precompilation-time/102617/1 "2023-08-08T21:43:03Z")

</div>

| Source of latency | Measurement method |
| --- | --- |
| Precompilation time | ??? |
| Load time | `@time_imports using MyPackage` |
| Compilation time | `@time MyPackage.myfunc()` |

I could do `@time Pkg.add("MyPackage")` but it combines the download and the precompilation. The documentation on [modules](https://docs.julialang.org/en/v1/manual/modules/#Module-initialization-and-precompilation) mentions `Base.compilecache(Base.identify_package("MyPackage"))`, is that the right command to time?

---

<div class="post-metadata">

**Author:** ![Benny](https://avatars.discourse-cdn.com/v4/letter/b/49beb7/32.png) [@Benny](https://discourse.julialang.org/u/Benny)\
**Post date:** [August 8, 2023, 10:18pm UTC](https://discourse.julialang.org/t/how-to-measure-precompilation-time/102617/2 "2023-08-08T22:18:14Z")

</div>

`compilecache` and those docs reads like that precompilation occurs during `using`/`import`, different from the precompilation reported when a package is added. I’m not sure what precompilation happens when, or even exactly what parts of the package is not included in the precompilation besides calls in ` __init__ `, but I’m sure there’s at least 2 times based on [this blogpost on precompilation](https://julialang.org/blog/2021/01/precompile_tutorial/):

> We’ll focus on _precompilation_,
> 
> ```julia
> julia> using SomePkg
> [Info: Precompiling SomePkg [12345678-abcd-9876-efab-1234abcd5e6f]
> 
> ```
> 
> or the related `Precompiling project...` output that occurs after updating packages on Julia 1.6

It’d be nice to figure out a timeline from package-adding to user calls of what inside a package is precompiled, compiled, and executed, as well as the ways we can measure the steps.

---

<div class="post-metadata">

**Author:** ![ufechner7](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ufechner7/32/51363_2.png) [@ufechner7](https://discourse.julialang.org/u/ufechner7)\
**Post date:** [August 9, 2023, 6:11am UTC](https://discourse.julialang.org/t/how-to-measure-precompilation-time/102617/3 "2023-08-09T06:11:34Z")

</div>

- close VSCode and all other programmes
- rename the .julia folder
- run `using Pkg; @time using <mypackage>`
- restart Julia and run the same code
- substract the second time from the first time
- delete the .julia folder and move back the original version

This gives you the precompilation time of `mypackage` and all dependant packages…  
It might include the download time, not sure how to avoid that.

Is that what you want to know?

---

<div class="post-metadata">

**Author:** ![jar1](https://avatars.discourse-cdn.com/v4/letter/j/c0e974/32.png) [@jar1](https://discourse.julialang.org/u/jar1)\
**Post date:** [August 9, 2023, 6:41am UTC](https://discourse.julialang.org/t/how-to-measure-precompilation-time/102617/4 "2023-08-09T06:41:59Z")

</div>

Is it not `Pkg.precompile(p)`?

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/27854_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [August 9, 2023, 7:00am UTC](https://discourse.julialang.org/t/how-to-measure-precompilation-time/102617/5 "2023-08-09T07:00:28Z")

</div>

I think those two times are the same, what makes you say they’re different? To me the difference is just the set of packages that get precompiled (the one you’ve added vs the ones necessary for the project that have not been precompiled yet / have changed since)  
[https://pkgdocs.julialang.org/v1/environments/#Environment-Precompilation](https://pkgdocs.julialang.org/v1/environments/#Environment-Precompilation)

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/27854_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [August 9, 2023, 7:00am UTC](https://discourse.julialang.org/t/how-to-measure-precompilation-time/102617/6 "2023-08-09T07:00:52Z")

</div>

I explored this option, which I called the nuclear option. Looking for something slightly less disruptive ^^

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/27854_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [August 9, 2023, 7:18am UTC](https://discourse.julialang.org/t/how-to-measure-precompilation-time/102617/7 "2023-08-09T07:18:38Z")

</div>

Indeed that looks promising, and I can even deactivate precompilation after download  
[https://pkgdocs.julialang.org/v1/api/#Pkg.precompile](https://pkgdocs.julialang.org/v1/api/#Pkg.precompile)

---

<div class="post-metadata">

**Author:** ![aplavin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/aplavin/32/222056_2.png) [@aplavin](https://discourse.julialang.org/u/aplavin)\
**Post date:** [August 9, 2023, 7:52am UTC](https://discourse.julialang.org/t/how-to-measure-precompilation-time/102617/8 "2023-08-09T07:52:23Z")

</div>

To measure precompilation or the total TTFX that includes precompilation, a scriptable approach is to create a temp DEPOT directory and pass it to Julia.  
Basically,

```julia
code = """
import Pkg
# calls to precompile etc
...
"""
projecttoml = """
[deps]
...
"""
mktempdir() do depot
    # if needed - copy registries:
    # mkdir(joinpath(depot, "registries"))
    # cp(expanduser("~/.julia/registries/General"), joinpath(depot, "registries/General"))
    # ...
    mktempdir() do env
        cd(env)
        write("Project.toml", projecttoml)
        write("script.jl", code)
        run(addenv(`julia --project script.jl`, Dict("JULIA_DEPOT_PATH" => depot)))
    end
end

```

See a full script at a gist of mine at [env\_benchmark.jl · GitHub](https://gist.github.com/aplavin/3f8490c7b944da86f9f1271c8cefaa7e). That specific script measures how long install + precompile + use will take if you create a new env with the same packages each 10 days (all updated deps will precompile).

---

<div class="post-metadata">

**Author:** ![mkitti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkitti/32/12459_2.png) [@mkitti](https://discourse.julialang.org/u/mkitti)\
**Post date:** [August 9, 2023, 8:48am UTC](https://discourse.julialang.org/t/how-to-measure-precompilation-time/102617/9 "2023-08-09T08:48:52Z")

</div>

Have you considered the following?

```julia
@time Base.compilecache(Base.identify_package("MyPackage"))

```

This is also compatible with `BenchmarkTools.jl`. Here is a demo with [MortgageCalculators.jl](https://github.com/mkitti/MortgageCalculators.jl).

```julia
julia> using BenchmarkTools

julia> @benchmark Base.compilecache(Base.identify_package("MortgageCalculators"))
[Info: Precompiling MortgageCalculators [cbda39b7-a5b8-49f7-bf50-e22af4f2d7a9]
[Info: Precompiling MortgageCalculators [cbda39b7-a5b8-49f7-bf50-e22af4f2d7a9]
[Info: Precompiling MortgageCalculators [cbda39b7-a5b8-49f7-bf50-e22af4f2d7a9]
[Info: Precompiling MortgageCalculators [cbda39b7-a5b8-49f7-bf50-e22af4f2d7a9]
[Info: Precompiling MortgageCalculators [cbda39b7-a5b8-49f7-bf50-e22af4f2d7a9]
[Info: Precompiling MortgageCalculators [cbda39b7-a5b8-49f7-bf50-e22af4f2d7a9]
[Info: Precompiling MortgageCalculators [cbda39b7-a5b8-49f7-bf50-e22af4f2d7a9]
BenchmarkTools.Trial: 2 samples with 1 evaluation.
 Range (min … max): 3.557 s … 3.676 s ┊ GC (min … max): 0.00% … 0.00%
 Time (median): 3.616 s ┊ GC (median): 0.00%
 Time (mean ± σ): 3.616 s ± 83.759 ms ┊ GC (mean ± σ): 0.00% ± 0.00%

  █ █  
  █▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁█ ▁
  3.56 s Histogram: frequency by time 3.68 s <

 Memory estimate: 311.12 KiB, allocs estimate: 3381.

```

Edit after reading more of the thread: The above measures the precompilation of the specific package, not it’s dependencies. It also invalidates any pre-existing cache that may exist.

If you to measure the total precompilation time, you may need to create a temporary Julia depot by setting `ENV["JULIA_DEPOT_PATH"]` or manipulating `DEPOT_PATH`. Even then, you may want to erase the path at `joinpath(DEPOT_PATH[1], "compiled")` before doing a `Pkg.precompile()`.

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/27854_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [August 9, 2023, 9:09am UTC](https://discourse.julialang.org/t/how-to-measure-precompilation-time/102617/10 "2023-08-09T09:09:14Z")

</div>

Does it assume that the dependencies are also precompiled? If not, do they get JIT-compiled and then discarded?

Very cool that it works with BenchmarkTools, is there any way to measure load time in the same way? As in, without restarting the session?

---

<div class="post-metadata">

**Author:** ![mkitti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkitti/32/12459_2.png) [@mkitti](https://discourse.julialang.org/u/mkitti)\
**Post date:** [August 9, 2023, 9:15am UTC](https://discourse.julialang.org/t/how-to-measure-precompilation-time/102617/11 "2023-08-09T09:15:29Z")

</div>

> [@gdalle](#):
>
> Does it assume that the dependencies are also precompiled? If not, do they get JIT-compiled and then discarded?

The dependencies will only get re-precompiled if there was some event to invalidate the compile cache. This is why you might want a temporary depot.

> [@gdalle](#):
>
> Very cool that it works with BenchmarkTools, is there any way to measure load time in the same way? As in, without restarting the session?

My guess is there may be a route via `Base.require` but looking at the implementation of Revise.jl may give another hint.

The other thought I have is is just running another Julia process. You can use `Base.julia_cmd()` for example.

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/27854_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [August 9, 2023, 9:39am UTC](https://discourse.julialang.org/t/how-to-measure-precompilation-time/102617/12 "2023-08-09T09:39:07Z")

</div>

> [@mkitti](#):
>
> The dependencies will only get re-precompiled if there was some event to invalidate the compile cache. This is why you might want a temporary depot.

What I mean is, I wondered whether the timing in this scenario includes precompilation of dependencies. And I guess this depends on the state of the depot