# A few basic questions about Precompilation

**URL:** <https://discourse.julialang.org/t/a-few-basic-questions-about-precompilation/71545>\
**Category:** General Usage\
**Tags:** precompilation\
**Created:** [November 16, 2021, 10:18am UTC](https://discourse.julialang.org/t/a-few-basic-questions-about-precompilation/71545 "2021-11-16T10:18:09Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Salmon](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/salmon/32/22968_2.png) [@Salmon](https://discourse.julialang.org/u/Salmon)\
**Post date:** [November 16, 2021, 10:18am UTC](https://discourse.julialang.org/t/a-few-basic-questions-about-precompilation/71545/1 "2021-11-16T10:18:09Z")

</div>

Hi everyone,  
I have recently tried with little success to add a precompilation step to my own package project and hope someone can help me clarify a few problems.  
In this post

> [@Understanding precompilation and its limitations (reducing latency)](https://discourse.julialang.org/t/understanding-precompilation-and-its-limitations-reducing-latency/48294):
>
> Many people have wanted to know more about how to reduce latency and the internal details of precompilation. I’ve just added a variant of the discussion below to SnoopCompile’s docs ([Snooping on inference: @snoopi · SnoopCompile](https://timholy.github.io/SnoopCompile.jl/dev/snoopi/#Understanding-precompilation-and-its-limitations-1)), but since it seemed to address an unmet need I decided to cross-post it here and invite discussion. Note that this concerns precompile directives in your package rather than usage of PackageCompiler, but it may help explain why precompile directives sometimes “work” an…

it is explained that one can precompile methods by calling `precompile(method, (types...)`, however making it work in practice seems quite cumbersome. In particular, it is pointed out that the compiler cannot really precompile functions called within `method` without further steps.

1. If I have a lot of functions is it really the only way to precompile them in a reasonable manner that I specify the argument types of each function individually?

2. If I have a function `exemplaryRun()`, which performs a typical workload with typical types, i can call this function within the definition of my module. From what I have seen, this executes the function only at the precompilation step. I would have expected that this would reduce compilation time so that methods only need to be compiled after loading the module if they have different types. Unfortunately, I noticed that somehow this introduces nasty run-time allocations when I use methods from my module, so somehow this approach does not seem to work. Why?

3. Is there an easy but relatively effective way to incorporate precompilation? I am aware of PackageCompiler but having to create a new sys-image everytime something is changed in the package is not really worth it as of now.

I hope there are some insightful answers to any of these 🙂

---

<div class="post-metadata">

**Author:** ![yoh-meyers](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yoh-meyers/32/14700_2.png) [@yoh-meyers](https://discourse.julialang.org/u/yoh-meyers)\
**Post date:** [January 4, 2022, 10:06am UTC](https://discourse.julialang.org/t/a-few-basic-questions-about-precompilation/71545/2 "2022-01-04T10:06:36Z")

</div>

I am facing same issue …

Unfortunately believe only way is to indeed call every function with each argument type. Then include output of `@snoopi_deep` via something similar to:

```julia
ccall(:jl_generating_output, Cint, ()) == 1 || begin
    for (root, dirs, files) in walkdir("precompile")
        for file in files
            include(joinpath(root, file))
        end
    end
end

```

Hope this enlightens you somehow.  
Let me know if you have any additional tips

---

<div class="post-metadata">

**Author:** ![tim.holy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tim.holy/32/52_2.png) [@tim.holy](https://discourse.julialang.org/u/tim.holy)\
**Post date:** [February 12, 2022, 8:31pm UTC](https://discourse.julialang.org/t/a-few-basic-questions-about-precompilation/71545/3 "2022-02-12T20:31:26Z")

</div>

The easiest way to force precompilation is to execute code in your package definition: just take whatever “work” you would have run under `@snoopi_deep` and just run it right before the final `end` of your package’s module. ImageCore.jl is a good example to look at.

There are a few cases where this isn’t possible, like when running the code requires an active display, or there would be other side effects. In such cases you still need `precompile`.

---

<div class="post-metadata">

**Author:** ![yoh-meyers](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yoh-meyers/32/14700_2.png) [@yoh-meyers](https://discourse.julialang.org/u/yoh-meyers)\
**Post date:** [February 16, 2022, 9:16am UTC](https://discourse.julialang.org/t/a-few-basic-questions-about-precompilation/71545/4 "2022-02-16T09:16:23Z")

</div>

Thank you!

---

<div class="post-metadata">

**Author:** ![Salmon](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/salmon/32/22968_2.png) [@Salmon](https://discourse.julialang.org/u/Salmon)\
**Post date:** [February 16, 2022, 2:08pm UTC](https://discourse.julialang.org/t/a-few-basic-questions-about-precompilation/71545/5 "2022-02-16T14:08:01Z")

</div>

thanks for your answer. I feel like I am missing something obvious here:  
Isn’t what you propose similar to my example (2.) where I run a function `exemparyRun()` in the definition of my module? For me this resulted in a crippling performance loss and many runtime allocations (which I still do not quite understand).  
I had a look at ImageCore.jl, but in the file `src/ImageCore.jl` there are only method and type definitions and I cannot see any code being executed, also not in the included files.

```julia
__precompile__ ()

module ImageCore

using Colors, FixedPointNumbers, MappedArrays, Graphics, ShowItLikeYouBuildIt
using OffsetArrays # for show.jl
using Colors: Fractional
[...]

"""
    rawview(img::AbstractArray{FixedPoint})
returns a "view" of `img` where the values are interpreted in terms of
their raw underlying storage. For example, if `img` is an `Array{N0f8}`,
the view will act like an `Array{UInt8}`.
"""
rawview{T<:FixedPoint}(a::AbstractArray{T}) = mappedarray((reinterpret, y->T(y,0)), a)
rawview{T<:FixedPoint}(a::Array{T}) = reinterpret(FixedPointNumbers.rawtype(T), a)
rawview{T<:Real}(a::AbstractArray{T}) = a

"""
    normedview([T], img::AbstractArray{Unsigned})
returns a "view" of `img` where the values are interpreted in terms of
`Normed` number types. For example, if `img` is an `Array{UInt8}`, the
view will act like an `Array{N0f8}`. Supply `T` if the element
type of `img` is `UInt16`, to specify whether you want a `N6f10`,
`N4f12`, `N2f14`, or `N0f16` result.
"""
normedview{T<:FixedPoint,S<:Unsigned}(::Type{T}, a::AbstractArray{S}) = mappedarray((y->T(y,0),reinterpret), a)
normedview{T<:FixedPoint,S<:Unsigned}(::Type{T}, a::Array{S}) = reinterpret(T, a)
normedview{T<:Normed}(::Type{T}, a::AbstractArray{T}) = a
normedview(a::AbstractArray{UInt8}) = normedview(N0f8, a)
normedview{T<:Normed}(a::AbstractArray{T}) = a

"""
    permuteddimsview(A, perm)
returns a "view" of `A` with its dimensions permuted as specified by
`perm`. This is like `permutedims`, except that it produces a view
rather than a copy of `A`; consequently, any manipulations you make to
the output will be mirrored in `A`. Compared to the copy, the view is
much faster to create, but generally slower to use.
"""
permuteddimsview(A, perm) = Base.PermutedDimsArrays.PermutedDimsArray(A, perm)

# Support transpose
Base.transpose{C<:Colorant}(a::AbstractMatrix{C}) = permutedims(a, (2,1))
function Base.transpose{C<:Colorant}(a::AbstractVector{C})
    ind = indices(a, 1)
    out = similar(Array{C}, (oftype(ind, Base.OneTo(1)), ind))
    outr = reshape(out, ind)
    copy!(outr, a)
    out
end

Base.ctranspose{C<:Colorant}(a::AbstractMatrix{C}) = transpose(a)
Base.ctranspose{C<:Colorant}(a::AbstractVector{C}) = transpose(a)

end ## module

```

I assume I am not looking in the right position or I misunderstood you. Could you clarify this for me?

---

<div class="post-metadata">

**Author:** ![tim.holy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tim.holy/32/52_2.png) [@tim.holy](https://discourse.julialang.org/u/tim.holy)\
**Post date:** [February 16, 2022, 2:12pm UTC](https://discourse.julialang.org/t/a-few-basic-questions-about-precompilation/71545/6 "2022-02-16T14:12:04Z")

</div>

For ImageCore, see [https://github.com/JuliaImages/ImageCore.jl/blob/ff7afc048c5832d8f3e6269f40ed06c4399cec7b/src/ImageCore.jl#L173-L176](https://github.com/JuliaImages/ImageCore.jl/blob/ff7afc048c5832d8f3e6269f40ed06c4399cec7b/src/ImageCore.jl#L173-L176) and [https://github.com/JuliaImages/ImageCore.jl/blob/master/src/precompile.jl](https://github.com/JuliaImages/ImageCore.jl/blob/master/src/precompile.jl).

> i can call this function within the definition of my module. From what I have seen, this executes the function only at the precompilation step. I would have expected that this would reduce compilation time so that methods only need to be compiled after loading the module if they have different types. Unfortunately, I noticed that somehow this introduces nasty run-time allocations when I use methods from my module, so somehow this approach does not seem to work. Why?

That’s probably [https://github.com/JuliaLang/julia/issues/35972](https://github.com/JuliaLang/julia/issues/35972), which should be fixed (and more) in Julia 1.8 (see [https://github.com/JuliaLang/julia/pull/43990](https://github.com/JuliaLang/julia/pull/43990)).

---

<div class="post-metadata">

**Author:** ![Salmon](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/salmon/32/22968_2.png) [@Salmon](https://discourse.julialang.org/u/Salmon)\
**Post date:** [February 17, 2022, 3:46pm UTC](https://discourse.julialang.org/t/a-few-basic-questions-about-precompilation/71545/7 "2022-02-17T15:46:10Z")

</div>

Thanks for the pointers, thats great!  
Also thanks for the link to the issue, I am looking forward to Julia 1.8 then :).