I’m putting this under First Steps rather than Performance, because I may be missing something trivial as a very new Julia user. The issue is that the code below takes about 15 seconds to run, when it is really doing nothing (the complete code file has more in it). The included file AnisotropyTools.jl has a few custom functions defined in it. My Julia install is brand new as of last night, so it is 0.6.2.
I am running the code from the command line on my Mac Book Pro (2016) as “julia file.jl”
I’d expect this code to be essentially instantaneous, yet it consistently takes about 15 seconds. Can someone help me understand why it takes so long?
Yes, this is expected and certainly not a good situation. You are simply running into the fact that Julia is recompiling your code every time you start it up. The plotting ecosystem involves a lot of code, so it tends to exhibit this issue more.
Try this: start Julia, copy in your code, wait for it to compile and run. Then, without closing Julia, run it again. You should see it run essentially instantly as you expected.
This will improve in future Julia versions, but for now I’d suggest modifying your workflow to do more in a single Julia session rather than restarting the Julia program frequently.
The main culprit here is probably Plots. Unfortunately the plotting tools (Plots.jl in particular at the moment) are designed in such a way that they suffer very badly from long compile times. There are two things you can do to alleviate this: first, I recommend using Gadfly.jl instead of Plots.jl for the time being as it tends to compile a little quicker. Second, you should use Revise.jl which allows you to load changes to your code without reloading the entire REPL. Unfortunately plotting in Julia is pretty impractical at the moment without the use of Revise.
Here’s a brief overview of the situation, in case this has given you serious concern: the developers so far have considered improvements to compile times a low priority. This is because compilation can be improved at any point in development without breaking anything, so there is no urgency in getting compile performance improvements in before 1.0. Furthermore, Julia’s compile caching is quite poor at the moment: the main reason that code is taking so long every time you start the REPL is because Julia is throwing away the majority of the binaries it compiled. At some point Julia will have full-blown compile caching (it’s been implied to me that this is planned for a 1.x release, but don’t quote me on that, I can’t speak for the devs). In the meantime, you might also want to check out PackageCompiler.jl, but as far as I know it’s still pretty experimental.
I have a bad experience using gnuplot in Fortran, because lines seem not smooth enough. Is there any other high performance plotting library available now? I really like Gadfly and the idea of grammar of graphics, but it is too slow.
I think your only real alternative is to fully embrace Revise.jl.
I’ve been using it by default for a couple of months now, it’s very reliable. There are a few caveats, but if you are doing plotting I think you’d be down to only a couple of restarts a day.
Depends on what you mean with “high performance” but I use https://github.com/KristofferC/PGFPlotsX.jl to generate plots for publication. Low compilation time (because LaTeX is doing all the real work).
Is something preventing Plots.jl from being able to be precompiled, or is it just so large that even with precompilation (which only “compiles” to the lowered AST, not even to LLVM IR or native code) it is slow?
I’m not familiar with the Plots.jl code base, but my understanding that it does contain an awful lot of code that doesn’t necessarily get precompiled, particularly for enabling the various back-ends (I think the use of Require might be a major culprit, but I’m not sure). It also has quite a few dependancies but so does Gadfly. So basically yes, it is my understanding that precompilation is particularly unhelpful for Plots. Remember also that the current Julia precompilation doesn’t precompile “everything”, so there usually some additional compilation involved when importing precompiled packages (not to mention your point about it only “compiling” to the lowered AST).
Thanks for the comments, and for the brief overview - I was in fact starting to be a bit disappointed with Julia! I really like the code I put together for all the right reasons, but the time was a bit disconcerting
I’ll be more optimistic now and revise my procedures for work arounds.
If you were very bothered by compile times, I’m imagining that you were coming from an interpreted language such as Python. These can be great for simple scripts, but they are abysmally slow for “real” code. For example, it would be folly to attempt to write a mixed integer linear optimizer in pure Python, but you certainly could do this in Julia. The cost is compile times, but in principle this is a very low cost, since it only needs to be paid once ever. In practice, usually what winds up happening is that you pay a few seconds up front in order to get a factor of 50 improvement after that.
It is unfortunate that Julia’s compile times are so slow right now, but this is not a fundamental limitation: the situation will improve. Sadly, plotting is the worst pain point right now by a very wide margin.
Personally I keep my Julia editor opened for as long as possible (up to a few weeks) and I also “warm it up” at startup by making a few plots automatically. I also keep a few consoles on the side that I can kill to test things out. It’s a bit annoying but manageable.
