TL;DR: When using PrecompileTools.jl and loading a package in Julia 1.6.7, allocations appear that are not expected in some functions. How can that be?
The issue:
I know that precompilation makes sense for Julia >=1.9. But still I would like to understand what happened in my code.
I added some precompilation directives in a package [here] and I have a bunch of tests in that package that check for allocations in some critical functions that must not allocate.
Now, the precompilation runs only for Julia >= 1.9, but initially it was running for all versions, and to my surprise, many functions that did not allocate in Julia 1.6 started to allocate, only after the precompilation directives where added.
This can be reproduced with that package by running the tests in Julia 1.6 after removing the if Base.VERSION >= v"1.9" from that precompilation workload.
So, I would like to understand how adding precompilation directives can affect the allocations (or any behavior for what matters) of the functions of a package. Is that known and only affects some Julia version prior to 1.9 (or any other version)?
Since precompilation runs even for older Julia versions, that can bring important performance regressions, and understanding what is going seems important.
I don’t know what the issue is but I think it got figured out and fixed for later Julia versions.
(Of course it wasn’t precompile tools back then but maybe simply related to running code at precompile time)
Maybe it would be reasonable for precompilation work (with PrecompileTools) to run only for Julia versions greater than those for which the issue is solved?
We can do that by hand, but not all packages have performance unit tests, and these things can pass if not properly warned.
Maybe. Clearly it needs to be installable on a wide variety of Julia versions, but it could be a no-op. I just didn’t feel comfortable making that choice for others because this is an issue that only sometimes occurs; other times, Julia 1.6 precompiles just fine. Since you can wrap the entire precompilation block in if Base.VERSION >= ..., I’d recommend doing it that way rather than having me impose a somewhat arbitrary choice on everyone.
I understand, and that’s what I did. But I wander if other packages that don’t have performance unit tests just didn’t notice the issue (I had one, and just added the conditional block).
From what I experienced, I would say that making the PrecompileTools macros do something by default only in Julia > 1.9 (and be an opt-in for older versions) would be better.
Anyway, it would be nice to have a warning in the manual of PrecompileTools concerning this issue. I’m not sure if a warning should be in the sense that precompilation can always have runtime performance impacts, or if that is restricted to Julia 1.8 or older, because of that resolved issue.
(that’s a parallel but related question: should every runtime change in behavior associated to precompilation be reported as bug? In Julia?)
Right, my point is that Person A (you) saw a slowdown. But Person B might see no slowdown and get a nice improvement in TTFX, using the same Julia version, with different packages. If we disable it for all Julia < 1.9, we take away Person B’s improvement to ensure that Person A doesn’t stumble into a runtime regression. I’m not say we certainly shouldn’t do it, but it makes me uneasy doing so.
(actually I started it before my previous answer, but sincerely, concerning what I can actually say about the issue and even where to put the note, probably it will give more work than less )
Concerning the general issue, I was under the impression that all this was 1.9+ oriented, and that any utility before that was a possible unwanted side-effect. Thus my suggestion that it could be disabled by default in previous Julia versions. But if that’s not the case, of course the decision become more complicated.
)