using LinearAlgebra, Plots, Distributions, Random;
Δt=0.01; T_sim=500; N_sim = 100;
time_sim = 0.0:Δt:T_sim
ε_sim = randn(MersenneTwister(123), N_sim, length(time_sim));
Z_sim =cumsum(ε_sim, dims=2)
plt_shock = plot(legend=:topright)
plot!(time_sim, Z_sim', lab="")
Plotting this simple simulation is insanely slow.
How can I speed it up?
What do you mean by insanely slow? The time to generate the first plot in a new session, or the time to generate any plot (after initial compilation)?
If the issue is the former, look into custom sysimages (or maybe try nightly now that native code caching has landed).
If it’s the latter, notice that you are plotting 5 million points if my math is right, which is bound to take some time. You can look at GPU accelerated plotting via WLMakie, or simply plot fewer points (there are only around 2m pixels on a 1,920x1,080 screen so you’re unlikely to distinguish 5m points anyway!)
I didn’t mean TTFP, I meant the latter.
There has gotta be a way for visualizations in Julia to realize I don’t need such a fine grid & speed it up…
You can try a different backend for Plots.jl to see which is faster, off the top of my head, I’m not sure which backend will be best for this task.
From my experience one of the Makie packages (i.e GLMakie or WGLMakie as @nilshg suggested) will probably be best if you want faster plots with millions of points, but this has a higher TTFP than Plots.jl.
Sysimages will help with the latency as well.
Do you really need to plot all those points? With that many ensembles, it is usually better to plot quantiles/means/etc.
I find using the SciML ecosystem is the easiest way to get things done and the plotting recipes with ensembles work great. And if you setup the brownian motion as a drift free SDE there are better algorithms for simulating them.
See Stochastic Differential Equations · DifferentialEquations.jl
It plots what you tell it to plot. If you want to plot less detail, it’s up to you to pass in a simplified data series.
But for good measure, is Plots at the most recent version? There were some serious slowdowns as a function of series number and series size as recently as a year ago.
Trying to put some numbers on the “insanely slow” observation.
On a Win11 laptop, with Julia 1.8.5 and latest Plots.jl and GR.jl:
Copying @jheinen, in case he can comment on the observed differences.
The graphical subsystems of the operating systems are overwhelmed with the representation of this number of points as lines (with typical attributes like line width, color, line type, line caps, etc.). At best, one could render the lines into an image in the simplest form with GR internal functions - but it would make more sense to aggregate the data:
using GR, Random
Δt=0.01; T_sim=500; N_sim = 100
time_sim = 0.0:Δt:T_sim
ε_sim = randn(MersenneTwister(123), N_sim, length(time_sim))
Z_sim = cumsum(ε_sim, dims=2)
shade(repeat(time_sim, N_sim), reshape(Z_sim', N_sim * length(time_sim)), colormap=-GR.COLORMAP_CMRMAP)
With this method plots can then be created efficiently in all output drivers (SVG, PDF, PS, image formats) - optionally also with the newly presented interaction options (zoom, pan, hover effects): GRDISPLAY=plot julia ...
I think the display speed is sufficient with this:
using BenchmarkTools
@btime shade(repeat(time_sim, N_sim), reshape(Z_sim', N_sim * length(time_sim)), colormap=-GR.COLORMAP_CMRMAP)
181.971 ms (159 allocations: 76.30 MiB)
@jheinen, thanks for your time and additional feedback.
(my initial trials with setting interactive zoom in GR failed in Windows, but that will be a separate thread)
The original post had 100 different curves, with different colors.
If we stick to that scenario, why GR is so fast (~1 s or so) compared to Plots.jl gr() backend? Is it autodecimating the data?
On another note, it would be very useful to have an additional keyword argument in Plots.jl plots() to turn automatic decimation on, or to plot big data at a user specified step (every n-th data point). There is a related issue open.
With GLMakie, I get the following:
julia> @time display(series(time_sim, Z_sim, color=rand(RGBf, 100); axis=(type=Axis, limits=(-10, 500, -600, 600))))
0.206328 seconds (221.90 k allocations: 109.205 MiB)
There seems to be a bug with limit calculation, which takes 6s, but if you supply the limits, its just 0.2s, and fully interactive 
You may also want to use finer lines with transparency:
series(time_sim, Z_sim, color=fill(to_color((:black, 0.03)), 100), linewidth=1; axis=(type=Axis, limits=(-10, 500, -600, 600))), transparency=true)
See also this thread: How to plot ~10 billion datapoint time series efficiently?