# Cache misses when 'using' packages since upgrading to 1.11

**URL:** <https://discourse.julialang.org/t/cache-misses-when-using-packages-since-upgrading-to-1-11/121445>\
**Category:** General Usage\
**Tags:** question, precompilation, cache\
**Created:** [October 18, 2024, 8:06am UTC](https://discourse.julialang.org/t/cache-misses-when-using-packages-since-upgrading-to-1-11/121445 "2024-10-18T08:06:42Z")\
**Posts on this page:** 15\
**Page:** 1

<div class="post-metadata">

**Author:** ![Eliassj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/eliassj/32/204117_2.png) [@Eliassj](https://discourse.julialang.org/u/Eliassj)\
**Post date:** [October 18, 2024, 8:06am UTC](https://discourse.julialang.org/t/cache-misses-when-using-packages-since-upgrading-to-1-11/121445/1 "2024-10-18T08:06:42Z")

</div>

Since upgrading to 1.11 I’ve been receiving messages about cache misses like this one:

```julia-auto
julia> using GLMakie
[Info: Precompiling GLMakie [e9467ef8-e4e7-5192-8a1a-b1aee30e663a] (cache misses: invalid header (8))

```

and this:

```julia-auto
julia> using Verify
Precompiling Verify...
  1 dependency successfully precompiled in 178 seconds. 606 already precompiled.
[Info: Precompiling Verify [156113c4-ddd8-477f-ab34-46faedf79717] (cache misses: include_dependency fsize change (1), wrong dep version loaded (1))

```

To clarify, precompiling works fine but when actually `using` packages the message about cache misses appear. And precompilation runs for a couple of minutes without any other messages. After this packages works as normal. The usual suspects are `include_dependency`, `fsize` and `invalid header`.

Has anyone else experienced something similar?

Versioninfo:

```julia-auto
Julia Version 1.11.1
Commit 8f5b7ca12ad (2024-10-16 10:53 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 8 × Intel(R) Core(TM) i5-1035G1 CPU @ 1.00GHz
  WORD_SIZE: 64
  LLVM: libLLVM-16.0.6 (ORCJIT, icelake-client)
Threads: 1 default, 0 interactive, 1 GC (on 8 virtual cores)

```

---

<div class="post-metadata">

**Author:** ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)\
**Post date:** [October 18, 2024, 8:41am UTC](https://discourse.julialang.org/t/cache-misses-when-using-packages-since-upgrading-to-1-11/121445/2 "2024-10-18T08:41:40Z")

</div>

@kristoffer.carlsson can correct me if I’m wrong but I think this is just additional information about stuff that was happening already anyway.

That is, when `using` a package sometimes Julia will find that a precompilation file created for that package can’t be loaded from the cache for some reason, and so it needs to re-precompile. This has always been the case, but in 1.11 Pkg tells you why it couldn’t use a cached precompiled file.

---

<div class="post-metadata">

**Author:** ![Eliassj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/eliassj/32/204117_2.png) [@Eliassj](https://discourse.julialang.org/u/Eliassj)\
**Post date:** [October 18, 2024, 9:32am UTC](https://discourse.julialang.org/t/cache-misses-when-using-packages-since-upgrading-to-1-11/121445/3 "2024-10-18T09:32:43Z")

</div>

That sounds reasonabIe. I should clarify that I felt like packages were taking longer to load/cache misses happening more often but that might just be a feeling since I’m being told about it now by Pkg.

---

<div class="post-metadata">

**Author:** ![giordano](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/giordano/32/2166_2.png) [@giordano](https://discourse.julialang.org/u/giordano)\
**Post date:** [October 18, 2024, 10:11am UTC](https://discourse.julialang.org/t/cache-misses-when-using-packages-since-upgrading-to-1-11/121445/4 "2024-10-18T10:11:56Z")

</div>

“invalid header” also probably sounds a little bit scarier than it was intended to be, maybe “incompatible header” would be a nicer wording.

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [October 18, 2024, 3:35pm UTC](https://discourse.julialang.org/t/cache-misses-when-using-packages-since-upgrading-to-1-11/121445/5 "2024-10-18T15:35:09Z")

</div>

Wouldn’t it just be better to not print anything by default? It is not that most user can do anything with that information anyway. It does give the impression that something is broken.

---

<div class="post-metadata">

**Author:** ![giordano](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/giordano/32/2166_2.png) [@giordano](https://discourse.julialang.org/u/giordano)\
**Post date:** [October 18, 2024, 4:20pm UTC](https://discourse.julialang.org/t/cache-misses-when-using-packages-since-upgrading-to-1-11/121445/6 "2024-10-18T16:20:16Z")

</div>

That’s not something that one who _does_ want to understand why precompilation is being triggered can retrieve afterwards.

---

<div class="post-metadata">

**Author:** ![brianguenter](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/brianguenter/32/29519_2.png) [@brianguenter](https://discourse.julialang.org/u/brianguenter)\
**Post date:** [October 18, 2024, 4:26pm UTC](https://discourse.julialang.org/t/cache-misses-when-using-packages-since-upgrading-to-1-11/121445/7 "2024-10-18T16:26:25Z")

</div>

I’ve also wondered about this message. It gives the incorrect impression that something has gone wrong, and there’s nothing the average user can, or should, do in response to the message. For beginning Julia programmers this will be confusing, and for experienced programmers it’s noise almost always.

An even more neutral phrasing, that doesn’t have any connotations of being in an error state, might be “rebuilding cache for xxx package, this might take a while”.

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [October 18, 2024, 5:25pm UTC](https://discourse.julialang.org/t/cache-misses-when-using-packages-since-upgrading-to-1-11/121445/8 "2024-10-18T17:25:08Z")

</div>

But couldn’t it be triggered by a julia command line argument?

---

<div class="post-metadata">

**Author:** ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)\
**Post date:** [October 18, 2024, 6:07pm UTC](https://discourse.julialang.org/t/cache-misses-when-using-packages-since-upgrading-to-1-11/121445/9 "2024-10-18T18:07:38Z")

</div>

I think that’s what it used to be.

Just from personal experience there’s a lot of hard-to-make-precise griping from people - myself included - about excessive recompilation. When it happens to me I usually (in the nicest possible way) blame Kristoffer, who reasonably asks me for a reproducer. The problem with that of course is that its not easy to reproduce these things, especially if I don’t want to nuke my compile cache.

So to the extent that this change will help find sources of unnecessary recompilation, I’m all for it.

---

<div class="post-metadata">

**Author:** ![jling](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jling/32/212909_2.png) [@jling](https://discourse.julialang.org/u/jling)\
**Post date:** [October 18, 2024, 6:08pm UTC](https://discourse.julialang.org/t/cache-misses-when-using-packages-since-upgrading-to-1-11/121445/10 "2024-10-18T18:08:13Z")

</div>

in that case you’d have to Ctrl-C to kill the current pre-compilation, and that may change the reason for the NEXT pre-compilation, so you still can’t know why it WAS pre-compiling, I think?

---

<div class="post-metadata">

**Author:** ![ianshmean](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ianshmean/32/216042_2.png) [@ianshmean](https://discourse.julialang.org/u/ianshmean)\
**Post date:** [October 18, 2024, 7:59pm UTC](https://discourse.julialang.org/t/cache-misses-when-using-packages-since-upgrading-to-1-11/121445/11 "2024-10-18T19:59:04Z")

</div>

Here’s a proposal to change “invalid header” to “incompatible header”.

> <https://github.com/JuliaLang/julia/pull/56240>
>
> Falling back to the older serial precompilation process is basically a bug (exce…pt for if a manifest hasn't been resolved) so https://github.com/JuliaLang/julia/pull/52619 added more info on why it's been hit so we have a chance of fixing issues that are otherwise very difficult to recreate.
> 
> However "invalid header" which usually just means it was made by a different julia version appears to sound too alarming to users. https://discourse.julialang.org/t/cache-misses-when-using-packages-since-upgrading-to-1-11/121445
> 
> So soften it there and in error messages, given it seems a better description.
> 
> Suggested by @giordano in https://discourse.julialang.org/t/cache-misses-when-using-packages-since-upgrading-to-1-11/121445/4?u=ianshmean

@Eliassj I suspect what’s happening in your case is indicated by `wrong dep version loaded (1)`. It seems like you’ve loaded a package from within another environment before trying to load `Verify` in a different environment, which has a common dependency/ies that differ in version to the current environment.

A classic way this happens is

```julia
$ julia
# startup.jl runs loading, say Revise, from your default env
pkg> activate MyProject
julia> using Verify
# Julia precompiles Verify, but only considering the active env, not the 
# default env higher in the env stack, but some deps are already loaded by Revise
#
# Julia tries to load verify that it just precompiled but because different 
# dep versions are loaded it cannot, so has to precompile new versions with 
# those versions, which it does in the serial precompile process starting with
# [ Info: Precompiling

```

It’s an unfortunate limitation of stacked environments. Ways to avoid it:

1. Always start up julia in the environment you are going to use, via `--project`, that way common dependencies will be the versions your env needs. i.e. avoid `pkg> activate`
2. Avoid using startup.jl to load packages if you’re going to be switching project after launch, and only load once you’re in the main env.
3. Use workspaces instead of the default env for common deps, which are coming in 1.12 [10. Project.toml and Manifest.toml · Pkg.jl](https://pkgdocs.julialang.org/dev/toml-files/#The-%5Bworkspace%5D-section)
4. The ability to unload packages is added to julia… or something like that… There is some work around that ongoing.

---

<div class="post-metadata">

**Author:** ![Ken\_Williams](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ken_williams/32/5289_2.png) [@Ken\_Williams](https://discourse.julialang.org/u/Ken_Williams)\
**Post date:** [November 19, 2024, 8:22pm UTC](https://discourse.julialang.org/t/cache-misses-when-using-packages-since-upgrading-to-1-11/121445/12 "2024-11-19T20:22:05Z")

</div>

Can someone clarify - if I don’t follow the advice of @ianshmean , and I do a bunch of `activate` switching on various projects, `use` of various packages in my `startup.jl` etc. etc., will I still get the correct versions of packages loaded, and the annoyance is just that I might get too much recompilation, which is a hassle?

Or is there a danger that I won’t even actually get the correct versions of packages loaded, if I’ve previously loaded something from the default env, say?

---

<div class="post-metadata">

**Author:** ![ianshmean](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ianshmean/32/216042_2.png) [@ianshmean](https://discourse.julialang.org/u/ianshmean)\
**Post date:** [November 20, 2024, 2:34pm UTC](https://discourse.julialang.org/t/cache-misses-when-using-packages-since-upgrading-to-1-11/121445/13 "2024-11-20T14:34:41Z")

</div>

With that pattern, yes you can end up with incompatible deps being loaded, without notice.

I tried to add a warning about this, but there were some edge cases that meant it needed redoing, which hasn’t finished yet [WIP: Reland "Warn if an already loaded package is attempted to be loaded from a different path" by IanButterworth · Pull Request #52798 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/pull/52798)

---

<div class="post-metadata">

**Author:** ![jling](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jling/32/212909_2.png) [@jling](https://discourse.julialang.org/u/jling)\
**Post date:** [January 21, 2025, 4:38pm UTC](https://discourse.julialang.org/t/cache-misses-when-using-packages-since-upgrading-to-1-11/121445/14 "2025-01-21T16:38:10Z")

</div>

> [@ianshmean](#):
>
> Always start up julia in the environment you are going to use, via `--project`, that way common dependencies will be the versions your env needs. i.e. avoid `pkg> activate`

I just want to add a note in this thread that another common way for people to hit this is via Jupyter notebook where the `IJulia.jl` was pre-installed somehow (for example, from Docker). And then in the first cell of the notebook users do:

```julia
Pkg.activate(...)

```

and you get a bunch of incompatible cache from there. In this case you can’t really avoid `]active` equivalent I think, as of 2024

---

<div class="post-metadata">

**Author:** ![Mend-Amar\_Badral](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mend-amar_badral/32/219656_2.png) [@Mend-Amar\_Badral](https://discourse.julialang.org/u/Mend-Amar_Badral)\
**Post date:** [March 27, 2025, 9:23am UTC](https://discourse.julialang.org/t/cache-misses-when-using-packages-since-upgrading-to-1-11/121445/15 "2025-03-27T09:23:48Z")

</div>

This IJulia bug happens even if I don’t import Pkg.