OK, I’d thought that the parsing and lowering passes consumed the bulk of the time (at least, given that that is what precompile is doing, and since things seem fairly snappy when things have been precompiled, that’s what it seemed).
Do you have any numbers on where time is being spent compiling Julia?
So much still to learn about the ins and outs of Julia (but it’s all fun!)
1 Like
Wow, in that case I’m glad I posted that reply. Over the years I’ve occasionally gotten a bit of grief that “julia’s compiler is slow because its parser is written in scheme”, which is not the case. It would be great if the front-end were the bottleneck and we could fix it just by rewriting that code in julia or C.
We have various ways of timing things and have done a bunch of profiling of the compiler. The breakdown depends a bit on what you compile, but generally the story is
isel time >= LLVM optimizer time > julia optimizer > type inference > front-end
The typical cause of precompile slowness is packages running code at load time, which requires codegen. If a package just contains, say, a bunch of method definitions and no top-level code then it should precompile pretty quickly. Of course, if we could save native code then running code at load time could become a good thing, but now it results in a double cost (both when precompiled and used). Plotting is a high-profile case, where despite precompilation it still takes a long time to get the first plot.
Another source of precompile slowness is that each package is compiled by a separate process, and each of those processes needs to load all dependencies of each package. That adds a small super-linear term to the time.
12 Likes
Point taken, and my apologies. My whole goal on this thread was to clarify the meaning of “meaning”, punning, and method merging with the hope of understanding why merging methods makes sense, what sorts of macros are reasonable, etc. I have a much better grasp of those issues than implementation details, and my questions apply to Julia as it is.
I don’t think anyone suggested that “no recompilation” in that method would take place, especially in code which is dispatching on abstract types or using higher-order functions. I think the thought was that, lets say Base.+, was already written for a bunch of Base based numbers , then if you wanted to create your own M.+ in your own namespace instead of in Base, then it could do the local compilation of M.+ on its own. Higher order functions in Base would also require recompilation, as you say. But that is my last comment on this. ADL is likely never going to happen, or at least in Julia v3.x. My concerns are more immediate. (though I really think you should look to see if there are any lessons from C++'s precompiled headers when you try code caching)
Whether a given method of + (or any function) can be separately compiled is not related to whether the method is part of Base.+. What matters is
- The dependencies of that method.
- Call sites that can see the method.
If a given a + b call site can see method X — either because it’s using Base.+ which contains X, or because X is callable via ADL — then compiling that call site requires information about X. This is really the key to the whole thing. For each call site you only need to know “which methods might this call site end up calling?” With ADL, the method would still be reachable from the same (or mostly the same) call sites, so the separate compilation story doesn’t change. (Note again here I’m only talking about separate compilation, not language ergonomics or other issues.)
4 Likes
Thank you for the explanation. I am still having trouble with thinking through the Julia compilation model because my brain is too tied to the C++ style “compiling only occurs once you have a complete set of concrete types for a given generic interface”. But besides my own interest on this stuff, I don’t think I have much to add on the organization of the compiler.
Back to language (or macro) ergonomics for me, after the hijacking of this “meaning” thread for compiler details (karma and a fair response to me hijacking the last compiler thread for philisophical dispatch concerns !)
Do I understand it right that instruction selection generates “Machine IR” that can also be cached? I don’t see native codegen in the list, so I guess MIR to native is really cheap?
Then a persistent second level cache would look quite attractive, and (at least conceptually) relatively simple (using cryptographic hashes to encode the world-state of the dependency-DAG, and simply LRU without any attempts of GC, ever).
I think doing semantically clever things is very hard, because a persistent cache needs to store different versions of foo(x::Int) depending on how dependencies like Base.:+ changed, in a possibly load-order dependent way, different sessions and package versions (developers churn out a lot of versions in a single sesssion!).
Can you point me to where (source file) the dependency DAG is currently handled? Unfortunately I don’t have a good overview yet 
1 Like
I did not say that constant recompilation is taking place … why do you suggest that @StefanKarpinski ?
It makes this discussion less productive.
I said that in the current definition of multiple dispatch it is impossible to gurrantee that code loaded and compiled on the spot. Will behave the same if loaded from the cache. See my example above.
This makes caching between sessions impossible without further “rules” and compromises to the system.
Regarding your qsort example. The MethodInstance for qsorting MyType belongs probably to MyTypes module.
So it should be cached there, however if every module imported potentially changes the definition of qsort , the binary cache for MyTypes is not valid without recompilation re inference etc.
Someone pointed out that if we take note on all The modules that is imported in a certain module, and “seal” the binary cache there then we are probably safe to assume that the cache will not change.
And I agree with that.
I have a POC of forward dispatch or more precisely context dispatch,on my github account under MultiFunctions.
I will later tonight , when in front of the computer , write a detailed explanation, if you are curious you can have a look now.
And yes it involves a Julia level linker.
I don’t know this part of the code, but I feel like I do because of this:
1 Like
I think doing semantically clever things is very hard, because a persistent cache needs to store different versions of foo(x::Int) depending on how dependencies like Base.:+ changed, in a possibly load-order dependent way, different sessions and package versions (developers churn out a lot of versions in a single sesssion!).
That seems like a fairly nice summary of some of the challenges someone might encounter if they wanted to resolve https://github.com/JuliaLang/issues/265 for example. But unfortunately, as you mentioned above, this “caching between sessions [is] impossible”.
But unfortunately, as you mentioned above, this “caching between sessions [is] impossible”.
Using a hash-tree (DAG), it is afaik possible, and I outlined a possible way above.
Yes, we now turned every hash-collision into possible remote code-execution; hence, the need for crypto-hashes, and possibly command-line options to freeze/disable the persistent cache. But you have the same issue when touching a dedup-enabled zfs volume.
Examples where it seemed to me that you are saying this:
Perhaps I’m misunderstanding what you’re saying but it seems a lot like your position is that the way method extension and dispatch works in Julia either prevents us from caching compiled code or causes us to have to recompile code all the time (or both). You don’t seem to have directly addressed the various technical points that @yuyichao, @jameson, @jeff.bezanson or myself have raised. It is of course possible that we have all misunderstood your position, but if so, then it would be helpful to clarify what your actual position is.
1 Like
I believe this is just false. But it depends somewhat on the details — perfect dependency resolution is in general undecidable. For example, it’s also impossible to “guarantee” that a makefile accurately reflects the dependencies in a project, and it’s also impossible to guarantee that a C header file matches a certain binary. But none of that makes separate compilation impossible. If you want separate compilation, you just do it anyway and ignore those problems. We could do that in julia too with appropriate compiler tooling.
Not true. In fact we already resolve dependencies with method-level granularity (in the fix for issue 265).
2 Likes
I always refer to caching between sessions. I apologize if it came out differently.
Julia runtime and design is genius , in my view,
Multiple dispatch and the type system changed the way I think… the simplicity is just beatifull.
And I want to see it overcome its infancy and become a major entity in the realm of software design and development.
And I believe I found a simple and elegant way to make Julia completely cacheable … so instead of depending on huge C/C++ projects with ccalls , this code could be a Julia module.
Currently this is not practical because of the startup time cost of first jitting large projects.
After everything is “warm” then the workflow is just amazing.
This is what I am targeting.
Regarding the runtime jitting compiling inference Base the type system , interfaces broadcasting comprehensions do-notation and more , you have created something remarkable.
1 Like
So your whole aim is to make caching compiled code between Julia sessions easier? That does clarify significantly. Thank you. It had seemed to me that you were talking about recompilation of code during a single Julia session.
I still think that you haven’t addressed @jeff.bezanson’s sum example, which really cuts to the heart of the matter.
if f is exported name of JeffModule or if ‘f’ has been specifically imported in my module then calling ‘f’ from MyModule will “see” sum with the methods available in Jeff’s Module and MyModule as if they were both importing a more basic version of sum.
and dispatch will dispatch according to standard rules
Ok, so the visible sum methods within f depend on who called f? That strikes me as much less compatible with separate compilation than what we do today. How can you statically compile JeffModule if it needs to do something different based on who calls it?
I’m not sure what this means. What does “basic” mean here?
I believe I found a solution to that … or at least a promising direction
calling ‘f’ from the context of MyModule is part of MyModule static compilation , not Jeff’s.
Module basic
function sum end
end
with MyModule and JeffModule declaring
import basic.sum
Yes, the call to f is part of MyModule, but the body of f is inside JeffModule. To me, separate compilation means that the body of f is compiled once and then left alone, and then anybody who wants to can call it. If the body of f has to be re-examined in any way when compiling MyModule, that’s no longer separate compilation.
usually in multiple dispatch you pass types that do not belong to a module ,you pass them through a module that do not know of them, manipulate them using some intermediate common reasoning , with actual values coming from the calling context that do know those types.
So anyway in that case the “door” or exported function ‘f’ is not meant to be cached at its defining module.
I think a good choice would be to say that if f(args…) is compilable and callable from module M then calling
that function should look no further.