How to precompile Makie?

question

#1

I tried to precompile Makie, because it seemed easy:

# add PackageCompiler
Pkg.add("PackageCompiler")
using PackageCompiler
# This is not well tested, so please be careful 

# The safe option:
PackageCompiler.compile_package("Makie", force = false) # can take around ~20 minutes

But I ran into issues.

  1. I tried to precompile in a project (not the default (v1.0) pkg>), where I installed Makie#master and PackageCompiler and ran the above commands, but it was missing both packages. It’s not a big problem, I may missed something about how PackageCompiler works.
  2. Then I moved to the default project and added every needed package (for Makie). One of my issues is, that it takes like an hour till I run into an Error, not really the 20 minute mentioned in the docs. It’s running on my laptop with i7-3632QM, SSD and 8GB RAM, so lacking resources should not be an issue. Does it take that much time, or am I doing somethong wrong?
  3. This is my main problem: while compiling, it is always missing some package, and as it takes an hour till I got an error, it is a really long time till I add every needed package. These are the packages that I had to install and wasn’t mentioned anywhere: ModernGL, MakieGallery, GLFW, ImageMagick, RData, Observables, RDatasets.

Julia and package versions:

julia> versioninfo()
Julia Version 1.0.2
Commit d789231e99 (2018-11-08 20:11 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: Intel(R) Core(TM) i7-3632QM CPU @ 2.20GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.0 (ORCJIT, ivybridge)

(v1.0) pkg> st
    Status `C:\Users\cstamas\.julia\environments\v1.0\Project.toml`
  [537997a7] AbstractPlotting v0.9.4 #master (https://github.com/JuliaPlots/AbstractPlotting.jl.git)
  [c52e3926] Atom v0.7.10
  [6e4b80f9] BenchmarkTools v0.4.2
  [336ed68f] CSV v0.4.3
  [be33ccc6] CUDAnative v1.0.1
  [5ae59095] Colors v0.9.5
  [a6e380b2] ControlSystems v0.5.0
  [150eb455] CoordinateTransformations v0.5.0
  [3a865a2d] CuArrays v0.9.0
  [1f15a43c] CxxWrap v0.8.1
  [a93c6f00] DataFrames v0.17.0
  [b4f34e82] Distances v0.7.4
  [4f61f5a4] FFTViews v0.2.0
  [7a1cc6ca] FFTW v0.2.4
  [5789e2e9] FileIO v1.0.5
  [587475ba] Flux v0.7.2
  [f7f18e0c] GLFW v2.3.0
  [e9467ef8] GLMakie v0.0.4 #master (https://github.com/JuliaPlots/GLMakie.jl.git)
  [28b8d3ca] GR v0.37.0
  [4d00f742] GeometryTypes v0.7.2
  [7073ff75] IJulia v1.16.0
  [6218d12a] ImageMagick v0.7.1
  [70c4c096] Indicators v0.4.0
  [c601a237] Interact v0.9.0
  [e5e0dc1b] Juno v0.5.4
  [b964fa9f] LaTeXStrings v1.0.3
  [ee78f7c6] Makie v0.9.1 #master (https://github.com/JuliaPlots/Makie.jl.git)
  [dbd62bd0] MakieGallery v0.0.2
  [931e9471] MechanismGeometries v0.1.2
  [283c5d60] MeshCat v0.5.0
  [6ad125db] MeshCatMechanisms v0.2.1
  [7269a6da] MeshIO v0.3.1
  [66fc600b] ModernGL v1.0.0
  [b8a86587] NearestNeighbors v0.4.3
  [47be7bcc] ORCA v0.2.0
  [9b87118b] PackageCompiler v0.6.1
  [f0f68f2c] PlotlyJS v0.12.3
  [91a5bcdd] Plots v0.23.0
  [c46f51b8] ProfileView v0.4.0
  [438e738f] PyCall v1.18.5
  [d330b81b] PyPlot v2.7.0
  [df47a6cb] RData v0.6.0
  [a223df75] Reactive v0.8.3
  [731186ca] RecursiveArrayTools v0.20.0
  [366cf18f] RigidBodyDynamics v1.3.0
  [6038ab10] Rotations v0.10.0
  [90137ffa] StaticArrays v0.10.2
  [2913bbd2] StatsBase v0.27.0
  [0f1e0344] WebIO v0.4.2

To sum up my questions: 1) do PackageCompiler run only in the “main” project? 2) Is it normal that it takes so much time to precompile? 3) Which packages are needed for Makie to precompile? (this is my main question)


#2

There’s a bug in the latest released version of PackageCompiler.jl with detecting test only dependencies, which I think is what you’re running into. I think that one is fixed on master, and if not, @sdanisch is probably on it :slight_smile:

I’d also heavily recommend to use compile_incremental(:Makie) instead, which is much faster and works with your existing sysimg instead of generating a new one from scratch.


#3

Thank you for the answer!
I tried with PackagCompiler#master and the results are the same. But I managed to add every dependency and the compilation ran succesfully. But it took several hours, which I don’t really understand (unless the compile time comment is outdated).
Also do I need to run any command to include the compiled image? Or it just works? (I didn’t try force=true yet.)


#4

You should try compile_incremental, which is quite a bit faster:
https://nextjournal.com/sdanisch/makie-static
You need to start julia with julia J path/to/image.
Or replace the current system image:

x = PackageCompiler.sysimg_folder("sys.so")
y = unsafe_string(Base.JLOptions().image_file)
cp(x, y, force = true)

Don’t do the above without backing up your current system image :wink: