Binary Caching and Pre-compilation .. again

[TsurHerman]

I want to present here again and perhaps more clearly, the concept of context dispatch , or more precisely resolving multiple dispatch based on some context.
I believe this can contribute to precompilation and load times issues, and may further the understanding of how to create a reusable binary cache for the compiled code and where and how to store such data.

The building blocks:

• A Context is a set of modules.

• The Context of module M is the set of all modules that module M depends upon.

• Extension: for the purpose of multiple dispatch , a module M can declare that it extends function fun from module B. This in “Julian” terms is referred to as module M specifically imports function fun from module B. Any call to M.fun will redirect to B.fun and B.fun itself will dispatch a to a specific method considering also the definitions of B.fun in its method table.

• Context Call: calling a function fun with a given context , the method table for fun will be determined by the modules that are present in the context (and that extend fun), the context is propagated recursively down the AST and every internal call is a context call.

There was previously some debate whether context call is feasible … I made some heroic attempts to prove so,
but @jrevels attempts were even more heroic, coming up with Cassette.jl which made my previous attempts redundant. kudos.

Binary Caching

Given a certain fixed context, it is quite clear that we can cache all compilation outputs for functions called within that context as long as we don’t define additional functions in any of the dependent modules.

It is also quite clear that the context of module Main is not cacheable, since it is dynamic and changing from invocation to invocation.

The current state of affairs in Julia is that every call is equivalent to a context call with the context of Main. Which makes binary caching very hard if not impossible.

Possible solutions
All solutions should incorporate some heuristics for “narrowing” the context, since any context that is not Main is probably cacheable.

• Narrow context based on types, calling function fun from module M with arguments types t1,t2…
  will narrow the context to the context of M if all types are instance-able from the context of M

• Post compilation narrowing: log the hierarchy of modules “visited” while compiling a function, log the compiled code in the smallest context that includes all visited modules. Then in a later stage if we encounter a function call from context M and there exist a binary cache for that function in M or in any of the modules inside Context of M … use the binary cache instead of compiling.

• other?

Further discussion

I feel this is a complicated subject better addressed in small pieces.

using Cassette.jl I can easily set up a POC for any kind of context dispatch rules, I will do so if there is interest from the devs and community.

[chakravala]

This is an important topic which needs to be discussed further.

The primary example I like to think of is the Reduce.jl package. Specifically, I encounter issues related to pre-compilation in conjunction with context. The package has an environment flag which can be used to toggle extra pre-compilation scripts ENV["REDPRE"].

These extra pre-compilation scripts are an experimental feature, intended to run some examples and tests of the code to ensure that some of the most used methods are called and compiled at least once.

Enabling the extra precompilation leads to a minor improvement in overall performance of the package when it is initially loaded.

However, one negative side-effect is INSTABILITY of the entire Julia program, which can lead to a segfault when Reduce is used in conjunction with another precompiled program (solved below).

julia> using Reduce,StaticArrays
[ Info: Recompiling stale cache file /home/flow/.julia/compiled/v1.0/Reduce/wEGBP.ji for Reduce [93e0c654-6965-5f22-aba9-9c1ae6b3c259]
[ Info: Precompiling extra Reduce methods (set `ENV["REDPRE"]="0"` to disable)
Reduce (Free CSL version, revision 4590), 11-May-18 ...

julia> Reduce.Algebra.:+(:(x+y),:z)

signal (11): Segmentation fault
in expression starting at no file:0
unknown function (ip: 0x7fad3a307d26)
unknown function (ip: 0x7fad3a30c92f)
...

The Solution to the instability, I discovered last night, is to simply disable the additional pre-compilation scripts.

After disabling this extra precompilation, all the mysterious issues disappeared. Now, I am also able to use StaticArrays in conjunction with Reduce to do symbolic computations on static dual grassmann numbers with the Grassmann.jl package,

julia> using Reduce, Grassmann
Reduce (Free CSL version, revision 4590), 11-May-18 ...

julia> basis"2"
(++, e, e₁, e₂, e₁₂)

julia> (:a*e1 + :b*e2) ∧ (:c*e1 + :d*e2)
0.0 + (a * d - b * c)e₁₂

julia> (:a*e1 + :b*e2) * (:c*e1 + :d*e2)
a * c + b * d + (a * d - b * c)e₁₂

The problem was that I had additional pre-compilation enabled by default for Linux users (an experimental feature with caching) and this caused a segfault. Disabling it by setting the ENV variable fixes that.

As @TsurHerman has pointed out, the context specific pre-compilation and caching continues to be a main issue with Julia going forwards. Many of the other important issues seem to have been worked out in Julia, but this context issue is something which continues to affect some users and developers.

Thus, I am also looking forward to further discussion on this topic.

[jpsamaroo]

So, maybe I’m missing something, but I’m not understanding how this:

@chakravala

However, one negative side-effect is INSTABILITY of the entire Julia program, which can lead to a segfault when Reduce is used in conjunction with another precompiled program (solved below).

is related to this:

@chakravala

As @TsurHerman has pointed out, the context specific pre-compilation and caching continues to be a main issue with Julia going forwards. Many of the other important issues seem to have been worked out in Julia, but this context issue is something which continues to affect some users and developers.

It sounds like you’re grossly conflating two different issues; namely, 1: Your precompile scripts lead to a segfault, and 2: Precompilation is slow, annoy, and needs fixing somehow.

If you want to solve #1, I would recommend that you first break apart what this script is actually breaking/segfaulting on; to be honest, that is a beast that I don’t think anyone will be able to help you fix, and so maybe you need to reconsider this approach. Just my 2 cents.

If you want to solve #2, then someone needs to actually do this:

@TsurHerman

using Cassette.jl I can easily set up a POC for any kind of context dispatch rules, I will do so if there is interest from the devs and community.

There have been quite a few threads started both here and on Julia’s Github issue tracker relating to this idea of “context dispatch solving binary caching/precompilation issues”, and frankly, I haven’t seen anything tangible actually come of it. There have also been rebuttals from a few of the core devs who do not believe that this approach has merit.

Of course, I am obviously not going to dissuade you from pursuing this approach if you so desire, and that’s fine. However, I think if we’re going to have a productive discussion on the benefits and downsides of this approach, the following items will need to be presented:

• A MWE of @TsurHerman 's ideas as actual working code
• Tests for this code that ensure that pre-existing Base/Stdlib/package code doesn’t break under this new scheme (or is easily fixable, with plausible suggestions for how to do this)
• Reproducible benchmarks of the time/memory/etc. savings of this new approach
• Optional but helpful: A plan for how to incrementally move from Julia’s current state of being, to this new approach, if we choose to do so

Now, I’m obviously just a naysayer that wants to put down people’s revolutionary and clever ideas. However, I think you’ll find that if you present me with clean, working code and benchmarks that illustrates your points in practice, I’ll very quickly come around and be willing to help out wherever I can

[jpsamaroo]

And for anyone who would like some “context” on this proposal and related discussions, here was everything I could find:

Julep: Taking multiple dispatch,export,import,binary compilation seriously

I want to formalize a julep for a design idea, I am asking the help of the community , please read and criticise freely. The problems it is addressing: Type piracy: It is too easy to break code in module M, even without directly evaluating code into it, by adding methods to Base functions or any other modules that module M depends upon. exmaple: import Base: + +(a::Float64,B::Float64) = 0.0 sum(rand(5)) 0.00 Conflicting names: if both module M and module N export the same function name f …

MultiFunctions: Context dispatch and binary cache

I have been writing this draft for several days , and got a little mixed up verbalising these ideas , which are simpler to understand using code and examples… If you want to jump straight into code, there is a link at the end. @Jameson @mbauman tl;dr There is a way to make Julia completely binary cacheable(between sessions), so it can become an effective way of developing and distributing .so .dll Background Function = a module wide constant symbol ( e.g :sum) without any specifications re…

"Meaning", type-piracy, and method merging

The meaning of “meaning” A common theme that comes up when discussing type-piracy/method merging, etc. is that there “can only be one meaning” for a given function without namespace qualifications (I am being careful there to say function, not method). I just want to get some clarification from the developers on this, because clarity in communication would help for educating users, deciding what macros are reasonable, and what can be discouraged. (Of course, part of this is driven by the fact…

Save actual machine code in precompile files

[chakravala]

@jpsamaroo

It sounds like you’re grossly conflating two different issues; namely, 1: Your precompile scripts lead to a segfault, and 2: Precompilation is slow, annoy, and needs fixing somehow.

It’s actually a third option, 3: you are uninformed about the details of my problem and what the causes of it are and how it relates.

@jpsamaroo

If you want to solve #1, I would recommend that you first break apart what this script is actually breaking/segfaulting on; to be honest, that is a beast that I don’t think anyone will be able to help you fix, and so maybe you need to reconsider this approach. Just my 2 cents.

Turns out I already did this exact testing myself, and already came to the conclusion that any amount of extra pre-compilation causes the issue. I don’t need anyone to do this work for me because I already investigated it.

The segfault is precisely related to the issues outlined in this thread. Specifically, there are extra pre-compilations happening with the script enabled which only occur in a specific context, especially with Base method extensions for example (which is a main component of the software providing extensions to many methods). Because the precompile cache is for a specific context, it segfaults when used with another package which extends methods also. It might be that they do not actually conflict with each other, but due to the precompiled binary caching (of the initially different extension context), it causes a segfault and a compatibility issue. Specifically, a function was compiled in a certain context ahead of time, and then called in a different context with different Base method extensions.

@TsurHerman

Given a certain fixed context, it is quite clear that we can cache all compilation outputs for functions called within that context as long as we don’t define additional functions in any of the dependent modules.

It is also quite clear that the context of module Main is not cacheable, since it is dynamic and changing from invocation to invocation.

The current state of affairs in Julia is that every call is equivalent to a context call with the context of Main. Which makes binary caching very hard if not impossible.

The solution is to simply not run the extra precompile script, which just means that the first time you run the program it takes a long time to precompile. However, if Julia had better context-specific binary caching of methods which get extended in multiple contexts, I believe this would help improve the same situation of mine… and this is what this topic was originally about: being able to cache method extensions handled in a context specific way.

[jpsamaroo]

@chakravala

It’s actually a third option, 3: you are uninformed about the details of my problem and what the causes of it are and how it relates.

I apologize, apparently I am indeed missing some context here. I did not mean to offend.

@chakravala

Turns out I already did this exact testing myself, and already came to the conclusion that any amount of extra pre-compilation causes the issue. I don’t need anyone to do this work for me because I already investigated it.

The segfault is precisely related to the issues outlined in this thread. Specifically, there are extra pre-compilations happening with the script enabled which only occur in a specific context, especially with Base method extensions for example (which is a main component of the software providing extensions to many methods). Because the precompile cache is for a specific context, it segfaults when used with another package which extends methods also. It might be that they do not actually conflict with each other, but due to the precompiled binary caching (of the initially different extension context), it causes a segfault and a compatibility issue. Specifically, a function was compiled in a certain context ahead of time, and then called in a different context with different Base method extensions.

Would you mind providing details on the specific issues you encountered while troubleshooting this issue? This could well be an issue in Julia that needs addressing (other than the obvious issue of precompile latency), and/or the specific details of your investigation might shed light on what specifically can be done to fix the proposed problem(s) with contexts.

[Kirby_Zhang]

it seems the longer binary caching is delayed the harder it becomes.
does Cassete.jl allow injecting binary results to the compiler, if someone wanted to experiment with per-environment/per-session caching?

[ToucheSir]

This thread was probably never updated because it’s so old, but there is active development towards native code caching in Support external linkage in "sysimages" by timholy · Pull Request #44527 · JuliaLang/julia · GitHub.