Roadmap for a faster time-to-first-plot?

[xiaodai]

Try Fezzik.jl

[lobingera]

maybe my 2c on some of this …

1 I’m now long enough here to have seen several recommended plotting solutions come and go. The problem of time-to-first-plot never was really bypassed.
2 The claims that GMT can do something other plotting packages cannot do (… try to do this in matlab …) are far-fetched. GMT has a sweet spot for projecting results onto maps which should not be surprising, because that’s GMT’s main job. The usability might be high here, but at the end-of-the-day julia needs a package/infrastructure that can plot into ANY given graphical context, may it be dedicated window, a tab of a browser, an OpenGL patterm etc. and afaics GMT has limitations (PS only?)
3 It’s really ironic that someone upthread proposes to build a dedicated plotting library in C for julia to be fast when one of julias main claims is: Julia is solving the two-language problem …
4 While a faster (or fast) code generation / compilation step is really appreciated, i think the way out (of problems like this for interactive REPL usage) is: code caching (between sessions) and similar compile-to-static. I got from Jeff’s lecture (“what is bad in julia”) the picture that julia will have more-and-improved-modularisation, so that a modules’ actions on data is fixed and not depending anymore on code called from/to other modules.

[mkborregaard]

@davidanthoff

I think Plots.jl is just especially bad at load times, it is just littered with design decisions that make it load slow, as far as I can tell. A bit of a shame that the main plotting package for julia is such a collection of bad patterns in terms of load times

I don’t think that’s true, and at the very least it would be nice to see your argumenting the basis of this negative statement. It’s slow loading (but otherwise really fast) because it’s written in Julia. Makie is even more so.

@Tamas_Papp

Plots.jl is not slow to start up because it is written in Julia, but because it contains a lot of conditionally loaded glue code.

This is just spreading false information, as Chris also remarked.

[Tamas_Papp]

@mkborregaard

This is just spreading false information, as Chris also remarked.

Apologies, my information was based on an incorrect understanding.

[affans]

@lobingera

3 It’s really ironic that someone upthread proposes to build a dedicated plotting library in C for julia to be fast when one of julias main claims is: Julia is solving the two-language problem …

Sure… but the interface would still be in Julia. For example, ggplot or vega can be interfaced with Plots.jl. I don’t the “two-language” problem means all code needs to be written in Julia. We already use a few C libraries for other stuff (Linear Algebra? Matrix computations?)

[ChrisRackauckas]

@affans

Sure… but the interface would still be in Julia. For example, ggplot or vega can be interfaced with Plots.jl. I don’t the “two-language” problem means all code needs to be written in Julia.

That still doesn’t handle the type-generic recursive argument pipeline, which is exactly what we want to have through something like Plots.jl but is causing the compilation issue. What makes us have automatic type handling and recipes is exactly what is causing over specialization. Interestingly, a blanket @nospecialize doesn’t help either, so it’s hard to see how to fix it. If you just want to call a C library and have no overhead, then you can already use GR.jl which is a good library for that, but of course it has to ignore types to do that.

@affans

We already use a few C libraries for other stuff (Linear Algebra? Matrix computations?)

Those are all specialized on Array{Float64}, Array{Float32}, and Array{Complex...}. Everything else has a generic Julia fallback or an error. We could do that with our plotting libraries (“error: every plot command has to end up at Array{Float64}s!”), but then of course plotting Symmetric{Measurements{Complex}} and stuff like that, which is right now handled automatically through the Plots.jl pipeline, would be a major pain.

[asprionj]

Fezzik.jl actually worked (as long as I didn’t do any plotting during the tracing period – causes some error during compilation since libGR cannot be found). Nice work!

[DNF]

@affans

I don’t the “two-language” problem means all code needs to be written in Julia.

The two-language problem is when you need to use a second language in order to get acceptable performance. So, what would you call this?

It’s not a two-language problem for those who call the library, necessarily, but it certainly is for those who write it, or want to contribute to it.

[Tamas_Papp]

@DNF

It’s not a two-language problem for those who call the library, necessarily, but it certainly is for those who write it, or want to contribute to it.

But ideally the boundary between the two groups should be easily permeable. (“Could have I feature X? Sure, make a PR.”)

[lobingera]

So Julia is solving a problem, that ‘ideally’ doesn’t exist?

[Tamas_Papp]

I am sorry, I don’t understand what you are saying here.

[lobingera]

I see.

[mbaz]

@Yifan_Liu

I completely rely on Gaston and UnicodePlots for visualization.

For plotting using text in the REPL, you can also use Gaston’s "dumb" terminal. It’s not exactly equivalent to UnicodePlots.jl because it uses only ASCII, but for simpler plots it might be good enough.

[Yifan_Liu]

Do you have an example code for this “dumb” terminal plotting?

[affans]

Thanks @ChrisRackauckas , I understand the situation more clearly now. I don’t really have any more suggestions for a Julia-only implementation. I only spoke from the perspective of an end user of Julia. To me it’s irrelevant whether the plotting comes from an external (already compiled) library or a Julia implementation. Plotting is really the only thing that I usually switch to R/ggplot to (i.e. export simulation data from Julia, plot in R). So for me the two-language problem still remains for me.

I agree, at a higher level, a Julia plotting solution is likely the best way forward. It will also allow some of us to even submit PR where we may not be able to do that with a C (or other language) library.

[bramtayl]

If we want a graphics package in Julia, then the question becomes what is the best open source graphics package, and how can we facilitate its translation into Julia. From the above, it sounds like the problem is overspecialization, though. So what is the solution to that?

[DoktorMike]

I love coding in julia and think it really does solve the two language problem for computing. For plotting, I still find myself firing up R and using ggplot. It’s just easier for me to get it right there and to do it fast. I don’t think it’s reasonable expecting julia to compete in the plotting space right now. That being said, I have high hopes for Makie and plotly in the longer run

[Zach_Christensen]

It’s probably hard to get a straight answer from my previous rambling post. I personally think that in the absence of any large financially backed project to create a graphics system from the ground up that our best bet is Makie. OpenGL is widely used and even some of the more progressive efforts in computer graphics are motivated to incorporate it somehow.

[mbaz]

@Yifan_Liu

Do you have an example code for this “dumb” terminal plotting?

julia> using Gaston
julia> set(terminal="dumb")
julia> t=0:0.05:1;
julia> plot(t,sin.(2π*t), size="50,20", title="A sine wave period")

                 A sine wave period               
                                                  
      +---------------------------------------+   
    1 |-+     ******* +       +      +      +-|   
      |     **       *                        |   
      |    *          *                       |   
  0.5 |-+  *           *                    +-|   
      |   *             *                     |   
      |  *               *                    |   
    0 |-*                 *                 *-|   
      |                    *               *  |   
      |                     *             *   |   
 -0.5 |-+                    *            * +-|   
      |                       *         **    |   
      |                        **     **      |   
   -1 |-+      +      +       +  *****      +-|   
      +---------------------------------------+   
        0     0.2    0.4     0.6    0.8     1 

The “dumb” terminal is one of gnuplot’s default terminal drivers – see page 216 in the current user manual.

[Palli]

@jeff.bezanson

In the meantime there are a couple tricks to try to work around latency issues:

1. Try running with -O0 or -O1
2. Try running with --compile=min
3. Try applying this patch:

@jeff.bezanson

You can already disable compilation by starting julia with --compile=min . We have actually always had an interpreter, which we use e.g. for simple top-level expressions. Yes, using it more

Four others in this thread quoted --compile=min. I’ve seen -O3 been 42% slower (and if I recall the default -O2 about as slow) than -O0 on short scripts (using one thread) and over 60% slower with 4 threads, not sure that comparison is valid (see benchmark game thread), and --compile=min way slower. -O0 may make most sense, at least for people on old machines as my decade old Core Duo laptop.