10-15 minute TTFP with Plots.jl... Please help

New to Julia
23

I might :broken_heart:. But this solution doesn’t generalize because there are other packages with precompile times that are too long to wait through daily or multiple times a day which do not have ready alternatives in other languages (e.g. Zygote/Flux)

I’ve added the following to my .julia/config/startup.jl file so that I can call _add("Plots"), wait for the precompilation, and then have a quick using Plots in any environment. Unlike adding Plots to my global environment, Plots will not reprecompile when it or any of its dependencies is updated unless I explicitly invoke _add("Plots") or _up("Plots"). When I switch Julia versions, it will reprecompile, but only once per julia version per call to _add or _up. Unlike the PackageCompiler route, with this approach the reprecompilation time on julia updates is lower and requires no manual intervention whatsoever. Normal Pkg usage should be entirely unaffected by this patch.

The implementation is mostly untested and the UX is terrible: 
# https://discourse.julialang.org/t/10-15-minute-ttfp-with-plots-jl-please-help/92636/23?u=lilith
if isinteractive()
    import Pkg

    function _add(pkg::AbstractString)
        project = dirname(Base.active_project())
        Pkg.activate(pkg, shared=true)
        Pkg.add(pkg)
        Pkg.activate(project)
        pkg ∈ KNOWN_PACKAGES && return
        push!(KNOWN_PACKAGES, Symbol(pkg))
        open(joinpath(@__DIR__, "known_packages"), "a") do io
            println(io, pkg)
        end
    end

    function _up(pkg::AbstractString)
        project = dirname(Base.active_project())
        Pkg.activate(pkg, shared=true)
        Pkg.update()
        Pkg.activate(project)
    end

    function _rm(pkg::AbstractString)
        filter!(≠(pkg), KNOWN_PACKAGES)
        open(joinpath(@__DIR__, "known_packages"), "w") do io
            for pkg in KNOWN_PACKAGES
                println(io, pkg)
            end
        end
    end

    isfile(joinpath(@__DIR__, "known_packages")) || touch(joinpath(@__DIR__, "known_packages"))
    const KNOWN_PACKAGES = Symbol.(readlines(joinpath(@__DIR__, "known_packages")))

    pushfirst!(Pkg.REPL.install_packages_hooks, symbols -> begin
        isdisjoint(symbols, KNOWN_PACKAGES) && return false
        if symbols ⊆ KNOWN_PACKAGES
            project = dirname(Base.active_project())
            for s in symbols # Install known packages
                Pkg.activate(string(s), shared=true)
                @eval using $s
            end
            Pkg.activate(project)
            @info "Successfully loaded $(join(symbols, ", ")). Please ignore the following error message:"
            true
        else
            @warn "$(intersection(symbols, KNOWN_PACKAGES)) are special packages that cannot be installed at the same time as ordinary packages $(setdiff(symbols, KNOWN_PACKAGES))."
            false
        end
    end)
end

The tradeoff is that I may, in theory, be exposed to known package version incompatibilities between the plotting stack and whatever else I may be working with. OTOH I think I get the same risk with PackageCompiler and updating the sysimg is harder than a call to _up("Plots")

Lovely! That was the problem. In 1.8 I got a sysimg made in 385 seconds which is in line with the ratio I’ve been seeing in this thread w.r.t. my timings vs other folks’. I’m still going to try the idea in this post first for the aforementioned reasons, though)

1 Like