I’m just going to expand slightly on my previous post.
It’s possible to be reasonably sophisticated in some aspects of programming, and yet be a ‘total rube’ in others. For myself, I’m comfortable with parametric types, type stability, generic programming, etc. etc. But I’m very uncomfortable with concepts such as package management, integrated testing, git wrangling, and userimg.jl
So even though someone might in some cases be literally looking at the screen and going “you have to show me which button to push”, they could still be capable of exploiting many of the strengths of Julia, and in fact be chafing against limitations in other languages.
Perhaps I’m going off-topic here (sorry about that). The relevance to this thread might be to highlight that even some “sophisticated” users can still use some hand-holding or need certain tools to be easier to use.
If the PackageCompiler path is a solution then ideally it should be made dead simple to use, having to know what userimg is or how to build Julia from source won’t do it for most people (for example I don’t know how any of these things, and I’m a relatively experienced user).
But from some of the comments above it looks like it’s not even clear if the PackageCompiler is a good solution, has someone made some systematic benchmarks ? For example compile Plots, DataFrames, Distributions, etc. and compare the timings for common use cases.
Just a couple of comments for people who are interested in this issue:
if you want to find out what’s taking the most time, it’s pretty easy to diagnose via profiling. For example, ProfileView.view(C=true) will tell you pretty clearly what fraction of the time is spent in inference vs LLVM.
if you’re using a pretty recent master/nightly of Julia, SnoopCompile now provides an easy way to measure inference time (@snoopi) as well as “core compilation” time (@snoopc).
One doesn’t have to wait for Jeff/Jameson/Keno to do something about this. For example, my measurements suggest that for packages I care about, much of the problem is due to the fact that some methods simply don’t get stored in the precompile cache file. There’s a start on a fix at https://github.com/JuliaLang/julia/pull/31466 ; I’ve been too short on time lately to finish it, but anyone comfortable with C has a shot at fixing this. Via a private email exchange, Jameson has kindly offered to tutor me through the final stages via google hangouts; I bet that if someone else from the community wanted to pick this up he’d extend the same offer.
We’re not talking about courses, which are in a controlled environment, but undergraduate/masters/PhD research projects. The students need to use their own computers, so “precompiled system image” is impossible. They also need to add and modify packages as their research dictates. They are also on a deadline, so latency issues are serious (to them).
It’s not systematic but I did manage to PackageCompiler Plots+GR, though with some headaches on the way (Plots+GR failed · Issue #215 · JuliaLang/PackageCompiler.jl · GitHub). It actually does solve the time-to-plot problem: using Plots is instantaneous (in the eyeball norm) and plot(1:10) takes only a second or so, which feels like booting up GKS.
All issues are “serious” to everyone affected. At this point, they can 1) live with the issue, 2) set up a precompiled image, 3) contribute to Julia to make this better.
I probably fit @DNF 's description of someone who knows a lot about a little and I think this sort of thing is the solution. We all have a lot of opinions based on a very diverse set of needs. I don’t think it’s realistic to expect the handful of very talented and intelligent developers to actually solve these problems in a way that is pleasing to everyone. I’d think we really need leaders in our respective fields to identify core issues in these fields and establish a lot of user friendly solutions. I don’t know if it’s possible for one catch all solution, but maybe it’s possible to have a handful of solutions that cover a limited set of problems individually.
What we ended up doing last year was setting up and tweaking a jupyterhub setup for the graduate students. Basically, bypassing any local installation at all so we could keep things stable, precompile a set of packages, etc… Right now an RA is trying to get this stuff built into the jupyterhub with the package compiler, which would help plenty. The environment is not really controlled, but acts as a stable baseline.
The students loved it, and it made julia far more accessible (i.e. these are students who are great at matlab but probably have opened up a terminal a half dozen times in their lives, and certainly have no idea what a dotfile is). That type of user is much more comfortable in a jupyter-only environment for a while, and it can take them pretty far.
Of course, this only works for an individual researcher if you university has the resources to run the hardware/support for a big jupyterhub installation.
Realistic or not, I do think it is possible to mostly solve this! We have not yet even attempted any of the known-possible solutions most likely to give huge latency improvements. This is not a situation where we are out of ideas. But, I do recommend that people invest in workarounds like custom system images in the short term.
You can already disable compilation by starting julia with --compile=min. We have actually always had an interpreter, which we use e.g. for simple top-level expressions. Yes, using it more is just the sort of thing we need to explore to slay the latency dragon.
If I understand correctly building custom system images smoothly also require some improvements in core julia.
I think a big obstacle for creating custom system images is __init__. It has to be executed for each package because it might be needed for executing the top-level statements of the downstream packages (e.g., using PyCall functions to compute some constants). However, __init__ may setup some global mutable state that is assumed to start from “empty” state (e.g., using push!). This assumption is invalid after the packages are compiled into the system image. It would be nice if --output-o supports loading from precompile cache files so that packages are loaded in the same way as normal usage. This way, __init__ does not have to be called in the Julia process with --output-o compiling the system image. IIUC this is not how it’s done at the moment (and that’s why PackageCompiler is manually calling __init__). Is it correct? If so, is it possible to change it?
If it is difficult/impossible or it does not make sense to slow down improving the compiler, I think some package authors need to learn some tricks to make their packages work with PackageCompiler (e.g., clear out mutable state in atexit).
This is a fairly relevant case, I think, and it is the case that I have experienced when teaching. First year grad students may be exposed to Julia in their course work. The things they will be doing are fairly simple, and are known ahead of time. If one could provide them with a system image that has the common tasks already compiled, or clear, easy to execute instructions to achieve this, they would get a more favorable initial impression. When they get to more innovative work later in their studies, they would already understand that occasionally needing to wait a bit for compilation is not that big of a deal. Actually, I really don’t care much about latency in my research work, I’m used to it. It gives me time to think about what I’m doing!
To throw another use case for low latency into the mix: serverless containers.
I have some scientific code that I put behind an HTTP server. The server is always up and I have to maintain it, which I’d rather not have to think about. It’d be nice to containerize the code and deploy to a serverless container solution such as Google Cloud Run. But this requires low latency. Rewriting the code in Python would cause an unacceptable drop in performance, and rewriting in C++ is more work than maintaining the existing server.
Seems like PackageCompiler.jl already does the job, which is great. This is just another data point for making the tool more usable.
Here’s an easy short term solution: prebuild Plots.jl and GR.jl in JuliaPro. This of course won’t solve my problems, but will solve 95% of the “new user” problems. Is that feasible?
The issue with this for new users is the nonstandard registry JuliaPro uses. It doesn’t take long before you start requiring students to use a package not in the registry, or at least at the correct version. Tried it and it was a disaster.
The JuliaPro registry contains all packages in General, not a subset. However, versions are frozen at a certain point. So newly tagged packages and versions are not available, until the the registry is updated. Updates happen every 2-3 months.
It is possible to switch JuliaPro to use the General registry. Once you switch, however you cannot (should not) switch back.
As I pointed out, it’s not that much - but may vary depending on your package.
You can test it like this:
>@time using Makie # or any other library
4.357982 seconds
>@time display(scatter(rand(6)))
35.448926 seconds
If you put using Makie into the userimg, the using Makie time will basically go to 0, but you’ll still need to compile any new function call to Makie.
So in this case, you’d get a speed up of ~1.14x - but note, that the more functions you call, the more gets compiled, the smaller the speed up will get
Calling 3 more Makie functions, I’m already at only 1.11x speed up.
It is not. Removing the conditional package loading in Plots does not help time-to-first-plot (we’ve tried), and implementing a native Julia plot library from scratch has been done - it’s called Makie and has a longer time-to-first-plot than Plots. This is a julia compiler thing, not a Plots thing.
Thanks for the explanation. I am aware of Makie, my understanding is that it is a very powerful software package that does very fancy 3D, utilizes the GPU, etc. I was thinking more along the lines of something with the capabilities of GR, but in native Julia. Just emitting eg SVG and leaving the rest to external programs.
It is not that simple though. I also thought the difficult part was the “rendering on the screen” part, but I’m no longer sure. I suspect part of the difficulty comes from having a pipeline that goes from potentially very different input to something that is “plottable”. Here for example I’m only loading AbstractPlotting, which is basically Makie without the OpenGL rendering and defaults to text rendering:
