I’m relatively new to Julia and and just diving into the waters of creating a package.
The Problem
The problem that I am having is that every time I do using MyPkg for the first time in the REPL the package takes a significant amount of time (~33 seconds) to load.
My current understanding
From reading through the documentation I thought that the idea of precompilation is that I essentially pay a one time penalty for compiling the package.
In return, as long as it doesn’t change, subsequent loading should be very quick.
Is this incorrect?
(Side note: This is where I understand Revise.jl comes in, essentially, in helping me avoid shutting down the REPL and thus having to recompile a package upon restart.)
A few other points/questions
This is does not appear to be the behavior I see with registered packages (e.g. JuMP). That is, these package load much more quickly.
I have this issue when I dev my package in the default environment (v1.5.3) as well as when I cd into the project folder and activate it with ] activate .
It is certainly possible (read: probable) that my package is not written optimally. Short of simply improving my code, if my understanding of precompilation is incorrect is there anything similar to what I’m describing that can be use (i.e. as long as the code does not change it does not need to go through precompile again)?
Is it saying “Precompiling MyPkg” every time, despite that you haven’t changed any of its code? That could be a bug or some subtle thing invalidating the precompilation cache. If its not saying “Precompiling MyPkg” but still taking 33s, its possible your package just takes a long time to load despite precompilation. Its not that hard to get to 33s if you have alot of using statements loading other complex packages. This is significantly improved on 1.6 compared to 1.5 fwiw.
I was afraid I’d get a response like this Unfortunately I can’t really do that (partially because the code is technically my employers and partially because it is a bit to big to share). That is why I added the bit about how the code itself is most probably not very optimal.
yes and no? it’s about 50/50 I would say. The impact on the loading time is not really significant though, which is further suggesting to me that the “problem” my simply be the contents of the project loading several things.
It’s also using a fair using calls. (this is a separate issue of code structuring that I’ve been reading a lot about here and still trying to wrap my head around)
If you could produce a MWE where you do using MyPkg from the REPL at it precompiles without warnings, then quit and restart your Julia session without changing any code then do using MyPkg again and it triggers the “Precompiling” message, then I think this would be pretty valuable since it definitely sounds like a bug in Julia at some level.
It could also be some subtle thing not technically considered a bug. Is your package or your ~/.julia on a network filesystem where maybe something is going wonky with file modification times? Or in some containerized environment? Or maybe you’re accidentally using two Julia versions without realizing it? If you run Julia 1.5.2 and precompile a package then run 1.5.3 and try to load it, it will trigger recompilation.
Difficult to know if you have the same issue, but I’ve come across such a problem when multiple REPL sessions are sharing the cache. I’ve got some distributed process running in one session and want to check on intermediate results in a second one—precompilation problems.
Thanks for the suggestion, but I don’t think that’s what’s going on in my case. I didn’t actually get as far as running anything just loading the package.
I’m afraid I have to apologiz. While working on producing a MWE, I realized that my original problem is not reproducing my original complaint
I suspect that @marius311 response is correct and I was confusing a long precompile time with a long load time (I apparently really need to work on cleaning up this code base).
I am sure the package was precompiled several times yesterday…but since I can’t reproduce it today I have to assume that I was conflating the few recompiles associated with some change I may have made and the, roughly similar in time, load without precompile.
Long story short: Thank you for your input and help, and apologies for what in retrospect was an ill placed question.
Unfortunately I can’t really do that (partially because the code is technically my employers and partially because it is a bit to big to share). That is why I added the bit about how the code itself is most probably not very optimal.
just loading the package.