# Precompiling module each time without any change

**URL:** https://discourse.julialang.org/t/precompiling-module-each-time-without-any-change/99711
**Category:** General Usage
**Tags:** question
**Created:** [June 1, 2023, 7:55am UTC](https://discourse.julialang.org/t/precompiling-module-each-time-without-any-change/99711 "2023-06-01T07:55:20Z")
**Posts on this page:** 13
**Page:** 1

<div class="post-metadata">

### Author: ![peremato](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/peremato/32/29128_2.png) [@peremato](https://discourse.julialang.org/u/peremato)
#### Post date: [June 1, 2023, 7:55am UTC](https://discourse.julialang.org/t/precompiling-module-each-time-without-any-change/99711/1 "2023-06-01T07:55:20Z")

</div>

Since I migrated to Julia 1.9 I get a pre-compilation of the module I am developing each time I start a test script. This was not the case with 1.8, for which I had sometimes to force a recompilation by modifying some of the files. Does anybody know how I can debug this to know what triggers the recompilation.

The module uses CxxWrap to interface to a C++ package. It is defined like this:

```julia
module Geant4

using CxxWrap
@wrapmodule("jlGeant4")

function __init__ ()
    @initcxx
end

end #module

```

---

<div class="post-metadata">

### Author: ![tim.holy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tim.holy/32/52_2.png) [@tim.holy](https://discourse.julialang.org/u/tim.holy)
#### Post date: [June 1, 2023, 2:06pm UTC](https://discourse.julialang.org/t/precompiling-module-each-time-without-any-change/99711/2 "2023-06-01T14:06:07Z")

</div>

You do have to compile once for normal usage and once when running `Pkg.test` because it sets `coverage=true`, and we compile different code for those two cases. (Turning on coverage slows down runtime performance.) But if you’ve just run `Pkg.test`, and you re-run it without changing your source code, it shouldn’t need to `Precompiling...` any packages.

The only things I can thing of are

- are you sure it’s precompiling? `Pkg` will “reinstall” the test dependencies of your project, but if they’re already installed and precompiled from the previous `Pkg.test` run, that will basically be a no-op (lots of printing, but very little time, and specifically no `Precompiling...`).
- might there be anything wrong with the clock on your system?
- are you changing the DEPOT each time? That will result in fresh compilation

> for which I had sometimes to force a recompilation by modifying some of the files

Sounds like you might benefit from `include_dependency`, if some files are required for your project but don’t fit within the definition of ordinary Julia code.

---

<div class="post-metadata">

### Author: ![peremato](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/peremato/32/29128_2.png) [@peremato](https://discourse.julialang.org/u/peremato)
#### Post date: [June 1, 2023, 2:36pm UTC](https://discourse.julialang.org/t/precompiling-module-each-time-without-any-change/99711/3 "2023-06-01T14:36:06Z")

</div>

> [@tim.holy](#):
>
> - are you sure it’s precompiling? `Pkg` will “reinstall” the test dependencies of your project, but if they’re already installed and precompiled from the previous `Pkg.test` run, that will basically be a no-op (lots of printing, but very little time, and specifically no `Precompiling...`).
> - might there be anything wrong with the clock on your system?
> - are you changing the DEPOT each time? That will result in fresh compilation

Yes, it is precompiling, at least is what is informing me.

```julia
Geant4.jl mato$ julia --project=. -i examples/basic/B2/B2a.jl 
[Info: Precompiling Geant4 [559df036-b7a0-42fd-85df-7d5dd9d70f44]

```

If I execute again immediately after without any change, I get the same INFO message.

> [@tim.holy](#):
>
> Sounds like you might benefit from `include_dependency`, if some files are required for your project but don’t fit within the definition of ordinary Julia code.

Thanks for the hint, which is now not needed until I resolve the current problem.

---

<div class="post-metadata">

### Author: ![peremato](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/peremato/32/29128_2.png) [@peremato](https://discourse.julialang.org/u/peremato)
#### Post date: [June 1, 2023, 2:50pm UTC](https://discourse.julialang.org/t/precompiling-module-each-time-without-any-change/99711/4 "2023-06-01T14:50:20Z")

</div>

BTW, this `Info` message only appears with the interactive option `-i`, but I’m sure it does the precompilation judging from the time it takes to start the script.

---

<div class="post-metadata">

### Author: ![ianshmean](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ianshmean/32/216042_2.png) [@ianshmean](https://discourse.julialang.org/u/ianshmean)
#### Post date: [June 1, 2023, 2:52pm UTC](https://discourse.julialang.org/t/precompiling-module-each-time-without-any-change/99711/5 "2023-06-01T14:52:45Z")

</div>

In non-interactive mode it’s a `@debug` so setting the env var `JULIA_DEBUG=Geant4` will make it show up

Edit: As @mkitti pointed out below this should be `JULIA_DEBUG=loading`

---

<div class="post-metadata">

### Author: ![mkitti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkitti/32/12459_2.png) [@mkitti](https://discourse.julialang.org/u/mkitti)
#### Post date: [June 1, 2023, 2:57pm UTC](https://discourse.julialang.org/t/precompiling-module-each-time-without-any-change/99711/6 "2023-06-01T14:57:56Z")

</div>

How is `Geant4` loaded? Have you modified the `LOAD_PATH` or set any environment variables? The results of `julia -e "using InteractiveUtils; versioninfo()"` would be informative here.

What is the output of `julia -E "DEPOT_PATH[1]"` ?

You might want to check `~/.julia/compiled/v1.9/Geant4` to see if there anything strange with the modified times of the files there.

---

<div class="post-metadata">

### Author: ![peremato](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/peremato/32/29128_2.png) [@peremato](https://discourse.julialang.org/u/peremato)
#### Post date: [June 1, 2023, 2:58pm UTC](https://discourse.julialang.org/t/precompiling-module-each-time-without-any-change/99711/7 "2023-06-01T14:58:41Z")

</div>

No, it does not show up.

```julia
Geant4.jl mato$ JULIA_DEBUG=Geant4 julia --project=. examples/basic/B2/B2a.jl 

**************************************************************
 Geant4 version Name: geant4-11-01-patch-01 [MT] (10-February-2023)
  << in Multi-threaded mode >> 
                       Copyright : Geant4 Collaboration
                      References : NIM A 506 (2003), 250-303
                                 : IEEE-TNS 53 (2006), 270-278
                                 : NIM A 835 (2016), 186-225
                             WWW : http://geant4.org/
**************************************************************...

```

---

<div class="post-metadata">

### Author: ![mkitti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkitti/32/12459_2.png) [@mkitti](https://discourse.julialang.org/u/mkitti)
#### Post date: [June 1, 2023, 3:00pm UTC](https://discourse.julialang.org/t/precompiling-module-each-time-without-any-change/99711/8 "2023-06-01T15:00:28Z")

</div>

Try `JULIA_DEBUG=loading`

---

<div class="post-metadata">

### Author: ![ianshmean](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ianshmean/32/216042_2.png) [@ianshmean](https://discourse.julialang.org/u/ianshmean)
#### Post date: [June 1, 2023, 3:03pm UTC](https://discourse.julialang.org/t/precompiling-module-each-time-without-any-change/99711/9 "2023-06-01T15:03:20Z")

</div>

Indeed, I got it wrong above. It will also tell you why any existing caches aren’t being used.

---

<div class="post-metadata">

### Author: ![peremato](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/peremato/32/29128_2.png) [@peremato](https://discourse.julialang.org/u/peremato)
#### Post date: [June 1, 2023, 3:08pm UTC](https://discourse.julialang.org/t/precompiling-module-each-time-without-any-change/99711/10 "2023-06-01T15:08:05Z")

</div>

> [@mkitti](#):
>
> How is `Geant4` loaded?

I am developing Geant4.jl, so my current directory is the checkout area. This is why I set the project `--project=.`.

> [@mkitti](#):
>
> Have you modified the `LOAD_PATH` or set any environment variables?

No

> [@mkitti](#):
>
> `julia -e "using InteractiveUtils; versioninfo()"` would be informative here

```julia
Geant4.jl mato$ julia -e "using InteractiveUtils; versioninfo()"
Julia Version 1.9.0
Commit 8e630552924 (2023-05-07 11:25 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin22.4.0)
  CPU: 8 × Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-14.0.6 (ORCJIT, skylake)
  Threads: 1 on 8 virtual cores

```

> [@mkitti](#):
>
> What is the output of `julia -E "DEPOT_PATH[1]"` ?

```julia
Geant4.jl mato$ julia -E "DEPOT_PATH[1]"
"/Users/mato/.julia"

```

> [@mkitti](#):
>
> You might want to check `~/.julia/compiled/v1.9/Geant4` to see if there anything strange with the modified times of the files there.

What I see strange is the date for the .dSYM files.

```julia
Geant4.jl mato$ ls -ls ~/.julia/compiled/v1.9/Geant4
total 152960
37624 -rwxrwxrwx 1 mato staff 19263064 May 31 18:55 GogkO_DkPlq.dylib
    0 drwxr-xr-x 3 mato staff 96 May 10 18:13 GogkO_DkPlq.dylib.dSYM
  136 -rw-r--r-- 1 mato staff 68134 May 31 18:55 GogkO_DkPlq.ji
37624 -rwxrwxrwx 1 mato staff 19263064 Jun 1 09:48 GogkO_d0CIK.dylib
    0 drwxr-xr-x 3 mato staff 96 May 10 21:50 GogkO_d0CIK.dylib.dSYM
  136 -rw-r--r-- 1 mato staff 68134 Jun 1 09:48 GogkO_d0CIK.ji
37624 -rwxrwxrwx 1 mato staff 19263064 Jun 1 16:57 GogkO_hxYin.dylib
    0 drwxr-xr-x 3 mato staff 96 May 10 18:04 GogkO_hxYin.dylib.dSYM
  136 -rw-r--r-- 1 mato staff 67987 Jun 1 16:57 GogkO_hxYin.ji
    0 -rw------- 1 mato staff 0 May 19 08:53 jl_HPGSxk
  136 -rw-r--r-- 1 mato staff 65953 May 16 10:52 jl_fTnfXe
    0 -rw------- 1 mato staff 0 May 16 10:52 jl_vW7QTQ
39544 -rw-r--r-- 1 mato staff 20244600 May 16 10:52 jl_wJrpie

```

---

<div class="post-metadata">

### Author: ![peremato](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/peremato/32/29128_2.png) [@peremato](https://discourse.julialang.org/u/peremato)
#### Post date: [June 1, 2023, 3:11pm UTC](https://discourse.julialang.org/t/precompiling-module-each-time-without-any-change/99711/11 "2023-06-01T15:11:54Z")

</div>

> [@mkitti](#):
>
> Try `JULIA_DEBUG=loading`

Great this helps!!

```julia
Geant4.jl mato$ JULIA_DEBUG=loading julia --project=. examples/basic/B2/B2a.jl 
┌ Debug: Rejecting stale cache file /Users/mato/.julia/compiled/v1.9/Geant4/GogkO_hxYin.ji because file /Users/mato/Development/Julia/Geant4.jl/gen/build/lib/libGeant4Wrap does not exist
└ @ Base loading.jl:2798
┌ Debug: Rejecting stale cache file /Users/mato/.julia/compiled/v1.9/Geant4/GogkO_d0CIK.ji (mtime 1.684766294152994e9) because file /Users/mato/Development/Julia/Geant4.jl/src/Geant4.jl (mtime 1.6856198847355766e9) has changed
....

```

The file that exists is `/Users/mato/Development/Julia/Geant4.jl/gen/build/lib/libGeant4Wrap.dylib` with the suffix.

What I am doing in the module is:

```julia
module Geant4
    using CxxWrap
    using Geant4_jll

    # Check whether the wrappers have been build locally otherwise use the binary package Geant4_julia_jll
    gendir = normpath(joinpath(@ __DIR__ , "../gen"))
    if isdir(joinpath(gendir, "build/lib"))
        include(joinpath(gendir, "jl/Geant4-export.jl"))
        @wrapmodule(joinpath(gendir, "build/lib", "libGeant4Wrap"))
    else
        using Geant4_julia_jll
        include(Geant4_julia_jll.Geant4_exports)
        @wrapmodule(Geant4_julia_jll.libGeant4Wrap)
    end
    ....

```

I do to fully specify the suffix for portability. Probably I should specify it depending on the platform I am.

---

<div class="post-metadata">

### Author: ![peremato](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/peremato/32/29128_2.png) [@peremato](https://discourse.julialang.org/u/peremato)
#### Post date: [June 1, 2023, 3:37pm UTC](https://discourse.julialang.org/t/precompiling-module-each-time-without-any-change/99711/12 "2023-06-01T15:37:56Z")

</div>

Problem solved. If I specify the full library name the precompilation is done only once on a change.

```julia
- @wrapmodule(joinpath(gendir, "build/lib", "libGeant4Wrap"))
+ @wrapmodule(joinpath(gendir, "build/lib", "libGeant4Wrap.$(Libdl.dlext)"))

```

However, I think this is a regression between 1.8 and 1.9. In [dlopen](https://gensoft.pasteur.fr/docs/julia/1.4.1/stdlib/Libdl.html#Libdl.dlopen) says that can be omitted.

---

<div class="post-metadata">

### Author: ![tim.holy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tim.holy/32/52_2.png) [@tim.holy](https://discourse.julialang.org/u/tim.holy)
#### Post date: [June 1, 2023, 3:54pm UTC](https://discourse.julialang.org/t/precompiling-module-each-time-without-any-change/99711/13 "2023-06-01T15:54:42Z")

</div>

If you could come up with a MWE that would be fantastic for fixing the regression. (We need a test to see if the problem has been fixed and stays that way.) Either way, it should be reported.
