Using (Web) packages together is much slower, e.g. 0.2 sec + 0.7 sec = 7.8 sec

I’ll first say, the composability of Julia packages is (otherwise) excellent.

I can e.g. use these two packages together, and these are the times, if loading individually (and exiting in-between):

julia> @time using WebSockets
  0.224824 seconds (135.01 k allocations: 8.546 MiB, 72.83% compilation time)
[restart]
julia> @time using WebIO
  0.693919 seconds (988.58 k allocations: 72.361 MiB, 2.00% gc time, 10.98% compilation time)

but together, as one using (or even two in succession, either order, I’ve before seen order matter):

julia> @time using WebSockets, WebIO
  7.769875 seconds (16.53 M allocations: 692.337 MiB, 6.33% gc time, 90.34% compilation time)

I think I know the reason (invalidations), but hat’s one of the biggest offender I’ve seen. I would like to know a good solution. I was looking into SnoopCompile[Core] and some other tricks I know of. If someone can fix (even only?) these two packages, then the Julia community will be grateful.

These are the dependencies of Blink.jl that I thought was being slow, but it’s not to blame nor any individual dependency (all relatively fast individually):

@time using Base64, BinDeps, Distributed, JSExpr, JSON, Lazy, Logging, MacroTools, Mustache, Mux, Reexport, Sockets, WebIO, WebSockets

I was looking into why the seemingly awesome (very recently registered package) CMPlot.jl is slow to start (13 sec. seemed excessive), because of PlotlyJS (8.5 sec) mostly it seemed, but actually it’s mostly “because” of Blink (8.1 sec).

One solution is:

$ ~/julia-1.7.0-rc3/bin/julia -O0 --compile=min

julia> @time using Blink
  1.384684 seconds (2.06 M allocations: 131.663 MiB, 5.32% gc time, 6.69% compilation time)

and similar for only:

julia> @time using CMPlot
  4.209774 seconds (6.11 M allocations: 392.835 MiB, 8.00% gc time, 2.60% compilation time)

Note, this isn’t about web programming [packages] per se. people may not realize they’re using web pages when “just” plotting (Blink brings in a full Chrome web browser, it’s just embedded local web browser and server).

This may all be bloated for disk space (I don’t really care), and people may blame the speed unnecessarily on that.

1 Like

I got “loading” these two packages from 7.7 sec. to 0.8 sec, by commenting out three lines, that I thought might break WebIO, but it doesn’t seem to.

Could the problem be these three lines, I disabled:

Requires is a Julia package that will magically make loading packages faster, maybe.

That’s a big “maybe”, anyone know why that is stated? Note, Blink has both packages as dependenices, and WebIO has WebSockets as a dependency and the example code for Blink still works. At runtime it was always slow at first (back when I last used it), and still is, maybe I just shifted the waiting from using to actually using the package.

Loading CMPlot is down to 6.1 sec. on defaults, and 3.8 sec on non-defaults.

WebIO loads a bunch of extra code conditionally when WebSockets is loaded. So it isn’t surprising that the load time is longer when they are loaded together in the same session.