How are compile cache files named?

Internals & Design
1

I’m using julia 1.5.3. This question is based on my curiosity of how julia works and not something that’s affecting my work.

I have several modules that I’ve written myself, and they in turn use some modules from the public registry. After I call precompile on my modules, I notice that there are multiple files in each module’s compile directory, but they all start with the same prefix. eg:

ubuntu@8b38e699432d:~/.julia/compiled/v1.5$ ls -l DataFrames/
total 16632
-r--r--r-- 1 ubuntu ubuntu 5678341 Mar 16 03:46 AR9oZ_LwnbX.ji
-r--r--r-- 1 ubuntu ubuntu 5693325 Mar 16 03:37 AR9oZ_op3w6.ji
-r--r--r-- 1 ubuntu ubuntu 5655448 Mar 16 03:52 AR9oZ_sVtHX.ji

My guess is that these are incremental compiles. For each module I’ve precompiled, if it uses different features of DataFrames, then those additional features are compiled into the new .ji file.

My questions are quite straightforward:

  1. Why the multiple filenames?
  2. Why do they all have the same prefix?
  3. How are the files named?

Thanks in advance for any insights.

2

Because if you use multiple environments then you can have multiple DataFrames installed at the same time, but you don’t want these to overwrite the same precompile file.

The function that names it is located here

The part before the underscore is determined based on the uuid and the version of the package.
The part after the underscore is based on the project, the sysimage, the julia binary and something called “preferences” (GitHub - JuliaPackaging/Preferences.jl: Project Preferences Package).

When loading a package, if an existing precompile file is valid, it will be used. Otherwise a new will be created, up to a maximum of 10 files. Then it starts to recycle.

2 Likes
3

I see, so if I activate a package and then precompile it, it generates a different precompile file than if I somehow precompile it from the default environment.

4

If the first precompile file generated is not valid for the one in the second environment, then yes.

5

I guess that confuses me. What do we mean by not valid? In all environments the Manifest.toml file lists the same version of (for example) DataFrames and all other required packages.

6

It means that something has changed with the package or its dependencies or julia etc. It shouldn’t need to recompile otherwise. To test, I created an environment with DataFrames and it precompiled. Then I copied that to another folder, and then it used the existing precompile file.

7

Ok, thanks. I’ll do some more tests and find out what’s happening.