Precompilation of developed packages

General Usage
1

Hello,

I have a project in development which is rather large, therefore resulting in long precompilation times when using it for the first time in a fresh Julia REPL session. The project can be separated in multiple independent modules with a clearly defined hierarchy, e.g. there are modules at the bottom level which form the basis for modules further up the hierarchy.

To reduce the precompilation time, I thought it would be a good idea to redefine the modules as packages. This allows me to dev a lower-level package from a local, relative path when using one from the upper hierarchy. I thought this would lead to significantly reduced precompilation times for the higher-level packages, however I found this to be false. Testing with a simple println command (as shown here: Tutorial on precompilation) confirmed that the lower-level packages are indeed precompiled each time when using a higher-level package.

My question is therefore: Does precompilation of packages only work for added packages, not for developed ones? I’d really like to keep the structure of having local, relative paths for the moment, how can I make this work with precompilation?

2

After trying quite a few things and reading a lot, I came up with the following solution which I want to document here. Apparently, it is indeed not possible to reuse precompiled .ji-files of packages added with dev.

The solution therefore is to make the local (unregistered) packages to git repos. To do that, make sure you can call git from the terminal of your choice (in my case, I used Windows PowerShell). If not, you need to install git first (Git - Downloads).

After that, make sure your package version is at least 0.1.1 (default is 0.1.0 when creating a new package). This can by checked and changed inside Project.toml.

Then, you need to init the git repo by navigating to your package directory and typing git init in the terminal. After that, add files / folders which you want to be version controlled by git add <file1> <file2> .... If you add a folder, all elements inside that folder are automatically added as well. After that, commit those changes by git commit -a -m "Your commit message here". You absolutely need to provide a commit message, otherwise the commit will fail! This workflow is derived from here: Routine use of git and github.

Then, you can add this unregistered package to another one by typing ] add "path/to/package/directory" inside the Julia REPL. If your unregistered package depends on another unregistered package, the following error can happen: Package Manager resolve complaining of "Unsatisfiable requirements" due to "no known versions"

The solution is to simply add the other package as well as a direct dependency.

Be aware that any changes to the unregistered package do not affect other packages anymore (as opposed to the dev approach). If you did some changes, you need to git commit -a -m "Your commit message here" again before the other package “sees” those changes.

This approach is probably as clunky as it gets, however I didn’t find a better way after searching and experimenting for overall at least a day. I don’t claim to know what I’m doing here, but I do know that the described approach works (for me) :slight_smile:

If there is any way to do this better, I’m always happy to learn.