PyPlot Continuing?

[iwelch]

I just tried PyPlot again. macos. I wanted to recreate the contour plots in

https://github.com/gizmaa/Julia_Examples/blob/master/pyplot_surfaceplot.jl

0.6

both the registered and master fail in 0.6.2. in the example with

julia> include("contourplot.jl")
ERROR: LoadError: PyError (ccall(@pysym(:PyObject_Call), PyPtr, (PyPtr, PyPtr, PyPtr), o, arg, kw)) <type 'exceptions.ValueError'>
ValueError(u"Unknown projection '3d'",)
  File "/Users/ivo/.julia/v0.6/Conda/deps/usr/lib/python2.7/site-packages/matplotlib/figure.py", line 1221, in add_subplot
    self, *args, **kwargs)
  File "/Users/ivo/.julia/v0.6/Conda/deps/usr/lib/python2.7/site-packages/matplotlib/projections/__init__.py", line 91, in process_projection_requirements
    projection_class = get_projection_class(projection)
  File "/Users/ivo/.julia/v0.6/Conda/deps/usr/lib/python2.7/site-packages/matplotlib/projections/__init__.py", line 65, in get_projection_class
    raise ValueError("Unknown projection '%s'" % projection)

Stacktrace:
 [1] pyerr_check at /Users/ivo/.julia/v0.6/PyCall/src/exception.jl:56 [inlined]
 [2] pyerr_check at /Users/ivo/.julia/v0.6/PyCall/src/exception.jl:61 [inlined]
 [3] macro expansion at /Users/ivo/.julia/v0.6/PyCall/src/exception.jl:81 [inlined]
 [4] #_pycall#67(::Array{Any,1}, ::Function, ::PyCall.PyObject, ::Int64, ::Vararg{Int64,N} where N) at /Users/ivo/.julia/v0.6/PyCall/src/PyCall.jl:658
 [5] (::PyCall.#kw##_pycall)(::Array{Any,1}, ::PyCall.#_pycall, ::PyCall.PyObject, ::Int64, ::Vararg{Int64,N} where N) at ./<missing>:0
 [6] #pycall#71(::Array{Any,1}, ::Function, ::PyCall.PyObject, ::Type{PyCall.PyAny}, ::Int64, ::Vararg{Int64,N} where N) at /Users/ivo/.julia/v0.6/PyCall/src/PyCall.jl:675
 [7] (::PyCall.#kw##pycall)(::Array{Any,1}, ::PyCall.#pycall, ::PyCall.PyObject, ::Type{PyCall.PyAny}, ::Int64, ::Vararg{Int64,N} where N) at ./<missing>:0
 [8] #call#72(::Array{Any,1}, ::PyCall.PyObject, ::Int64, ::Vararg{Int64,N} where N) at /Users/ivo/.julia/v0.6/PyCall/src/PyCall.jl:678
 [9] (::PyCall.#kw#PyObject)(::Array{Any,1}, ::PyCall.PyObject, ::Int64, ::Vararg{Int64,N} where N) at ./<missing>:0
 [10] include_from_node1(::String) at ./loading.jl:576
 [11] include(::String) at ./sysimg.jl:14
while loading /Users/ivo/bitcookbook/contourplot.jl, in expression starting on line 37

0.7

more worrisome is that PyPlot completely dies (deprecations, segfaults, etc) in julia 0.7. my understanding is that packages should be brought up to 0.7 in preparation for the impending 1.0 release.

given that PyPlot has not made progress on this path, is it continuing, or is it going the way of the dodo?

[ChrisRackauckas]

@iwelch

more worrisome is that PyPlot completely dies (deprecations, segfaults, etc) in julia 0.7. my understanding is that packages should be brought up to 0.7 in preparation for the impending 1.0 release.

Not yet. There’s no reason to expect this. It’s not even alpha yet. Hell, the entire structure of broadcast just changed on v0.7 this morning. It’s not stable and not for general use. Yet.

[stevengj]

The error on 0.6 is just because you haven’t installed/loaded 3d plotting in Python—it’s not a bug.

(Unfortunately Anaconda doesn’t install the 3d plotting by default with matplotlib these days, due to some dependency conflict in the basemap package IIRC, but you can install basemap manually if you don’t mind Conda downgrading some versions. See: Error installing PyPlot: The Python package mpl_toolkits.mplot3d could not be found by pyimport - #13 by stevengj)

As for 0.7, it will eventually happen but 0.7 is not suitable for general use at this point. PyCall itself only works on 0.7 if you have master versions of other packages (specifically MacroTools), and was only just fixed again today (due to recent changes in 0.7).

[stevengj]

Also, you should read the PyPlot manual on 3d plotting: https://github.com/JuliaPy/PyPlot.jl/blob/ac1827e6ac53e2a89e03e1f1ebc8474866701527/README.md#3d-plotting

In particular, if you manually construct an axis object with projection=“3d” rather than using the provided 3d plotting functions, the manual explains that you need to call using3D() to load Matplotlib’s 3d plotting module.

(This wasn’t the case in older versions, which loaded the 3d-plotting module on startup, but that broke when Anaconda stopped shipping mplot3d by default.)

[iwelch]

Hell, the entire structure of broadcast just changed on v0.7 this morning.

I guess it isn’t this close to release of 1.0 as I thought.

you haven’t installed/loaded 3d plotting in Python—it’s not a bug

it may not be a bug that I did not install the python package, but maybe the julia package should install whatever is needed?! I would suggest that it qualifies as a buglet that neither the installer nor the “using PyPlot” nor the actual execution issued a useful warning message…“please install blah blah first”.

I don’t use python and I don’t know what anaconda is (or how to install it on macos) or matplotlib. is familiarity with this stuff a prerequisite to reasonable use?

of course, iI am just offering a suggestion for a package in development, and I am glad to hear it is continuing.

/iaw

[stevengj]

@iwelch

it may not be a bug that I did not install the python package, but maybe the julia package should install whatever is needed?!

One problem is that (last I checked) this involves downgrading some Anaconda packages, so I’m reluctant to do it automatically — that should really be fixed in Anaconda and/or matplotlib/basemap. Another problem is the (apparent?) difficulty of monkey-patching Matplotlib to intercept all places projection="3d" can be passed, although I haven’t looked into that in detail.

I would suggest that it qualifies as a buglet that neither the installer nor the “using PyPlot” nor the actual execution issued a useful warning message…“please install blah blah first”.

Note that you would get exactly the same error message in Python. If you find this error message obscure, you should file an issue with Matplotlib itself.

PyPlot.jl already tries to make some things nicer than they would be in Python, e.g. it automatically installs Matplotlib itself if you are using Conda, and is somewhat more flexible about argument types in the 3d plotting functions, but there is a limit to how far it is practical to go in intercepting Python library calls and improving their behavior. I’m especially reluctant to start monkey-patching Python just to improve their error messages.

If we can figure out a single entry point to cleanly intercept all projection="3d" options in Matplotlib, I would certainly consider patching it to call using3D() to auto-load mplot3d. (Though it would be better to fix this in Matplotlib itself.)

[iwelch]

same error message in Python

I had a choice: learning Julia or learning Python. I went with Julia. I hope I won’t have to learn both. ;-).

Actually, I think clear informative errors when any Python related problems are detected, whether they be lack of installation of any Python or problems on their backend, would be all that’s needed. just my two cents.

I plan to try PyPlot again in a few months and see if its basic (not advanced!) use has become “seamless.”

regards,

/iaw

[stevengj]

@iwelch

I had a choice: learning Julia or learning Python. I went with Julia. I hope I won’t have to learn both. ;-).

If you are calling Python libraries from Julia, you generally need to understand at least a little of both. (For one thing, all of the Matplotlib documentation is in Python.)

If you want a less Pythonic interface, you could try something like Plots.jl (which can use PyPlot as a backend). In general, the intention of PyPlot is to be a nearly 1-to-1 map to the Python interface, leaving it to other packages to layer other interfaces on top if desired.

[tkoolen]

@iwelch

I guess it isn’t this close to release of 1.0 as I thought.

This has been in preparation for quite a while and as I understand it, this is second to last major change before 1.0. From 1.0 progress/status (March 16):

@StefanKarpinski

While that’s going on, @mbauman has been making a series of PRs improving generic array code and broadcasting. Every PR that Matt makes is, in my mind, another a significant chunk of array brokenness that we would eventually have regretted that we now won’t have to live with until Julia 2.0. So I’m glad the optimizer work is giving him some time to get it done before 1.0-alpha.

Also note that the original PR was opened Jan 3, https://github.com/JuliaLang/julia/pull/25377.

[iwelch]

I was hoping PyPlot would develop to the stage where its use would become seamless. the same way that the average LaTeX user does not need to worry about pdf internals. of course, I understand that everything in julia land is still early.

I think you are right that it’s reasonable to point to at Python docs to find out what calls and plotting options are available. more importantly, pyplot has wonderful julia examples at https://gist.github.com/gizmaa/7214002. This can substitute for a lot of the details. it also shows how to translate options.

alas, I think I do have one reasonable point. I think asking julia users to understand the python ecosystem is a bit too much. I did not even know what matplotlib was. I do not know how python is installed on macos. or what anaconda is?! or how there are regressions somewhere in a python package. I am ok with being asked to install stuff, too. but this is a lot easier if there are clearer error messages with instructions what needs to be done (not what needs to be figured out).

well, let’s see when 1.0 will happen. I am curious.

/iaw

[tkoolen]

I’m sympathetic to your point regarding plotting. I’ve had my share of frustrations with plotting as well (several this week, in fact). I also understand that for a lot of users, plotting is absolutely integral to their experience with the language (for me, it’s important, but there are definitely a lot of days where I don’t need to plot anything at all). However, I do think that the plotting situation is not very strongly tied to Julia 1.0, possibly apart from the new possibilities that Pkg3 will bring.

I’m personally hoping that a pure Julia solution will win over PyPlot/GR and the like in the end, and (possibly controversially), I’m not 100% convinced that a ‘backend-agnostic’ solution like Plots.jl is the best solution in the end.

[ChrisRackauckas]

@iwelch

I was hoping PyPlot would develop to the stage where its use would become seamless. the same way that the average LaTeX user does not need to worry about pdf internals. of course, I understand that everything in julia land is still early.

IIUC, and as @stevengj just said, that’s not the point of this package. You’re asking for PyPlot to be Plots.jl instead of using Plots.jl? They have tradeoffs, but the nice thing about PyPlot is how close to the Python API it is so that way you can grab every little feature. If that’s not what you’re using it for, you’re using the wrong library.

@tkoolen

I’m not 100% convinced that a ‘backend-agnostic’ solution like Plots.jl is the best solution in the end.

I think a lot of people are looking at Makie.jl as the solution, and it really is coming along nicely.

[iwelch]

if I understand you correctly, then PyPlot is really more for people who come from Python and want to use a julia package that is very similar to their existing base, rather than intended to become a primary graphics go-to package for native ju(ve)liers without Python use.

thanks for the pointer to Makie.jl. I will put it on my list of packages to look into.

my primary needs are actually more for a production quality than for a real-time plotter. something to allow me to leave the last vestiges of R, which has really nice production plotting. what drove me to PyPlot were the ready-to-go examples. when Plots.jl has this, too, it will probably be good enough.

/iaw

[ChrisRackauckas]

@iwelch

if I understand you correctly, then PyPlot is really more for people who come from Python and want to use a julia package that is very similar to their existing base, rather than intended to become a primary graphics go-to package for native ju(ve)liers without Python use.

Basically. And matplotlib is quite a large package with intricate details, so if you want to be able to use “the whole thing” you need to be able to call any of its functions.

@iwelch

my primary needs are actually more for a production quality than for a real-time plotter. something to allow me to leave the last vestiges of R, which has really nice production plotting. what drove me to PyPlot were the ready-to-go examples. when Plots.jl has this, too, it will probably be good enough.

Plots.jl has it. In fact, you can use PyPlot as the backend and save to PDFs for papers. Here’s a bunch of examples:

http://docs.juliaplots.org/latest/examples/pyplot/

These days, I’d say that unless you’re doing something funny with quiver plots or finite element trisurf plots then you’ll be fine with Plots.jl for standard publication plots with a Julia API and Julia documentation.

[Tamas_Papp]

@tkoolen

I’ve had my share of frustrations with plotting as well (several this week, in fact).

I was teaching an intensive course this week, and made 50+ plots for various slides on MCMC & other Bayesian numerical examples using Julia with PGFPlotsX.jl. It was absolutely seamless with surprisingly little frustration, and integrated nicely into LaTeX without any extra work.

[tkoolen]

Yeah, I should look into PGFPlotsX again. My first test a couple of weeks ago seemed promising. Thank you for your efforts. I’m interested in both ‘production quality’ and ‘real-time’, as @iwelch put it, so both Makie and PGFPlotsX are good developments in my view.

[stevengj]

@iwelch

if I understand you correctly, then PyPlot is really more for people who come from Python and want to use a julia package that is very similar to their existing base

That, but also to give you the full power of Matplotlib. The purpose of PyCall and PyPlot is to take advantage of the years of development and vast resources that have gone into (and continue to go into) Matplotlib and other Python libraries.

The reality is that, for a long time, there will be more Python libraries than Julia libraries, and the Python libraries will often have features that their pure-Julia counterparts lack. Plotting is a great example of this — it takes years to develop a full-featured library because of the enormous number of features that are expected for publication-quality plots. Having access to the entire functionality of Matplotlib requires a somewhat Pythonic interface.

I had a choice: learning Julia or learning Python. I went with Julia. I hope I won’t have to learn both.

No matter what language you learn, it is almost always beneficial to learn other programming languages too, at least at a basic level. I have literally never known a good programmer who knew only one language.

[iwelch]

@stevengj

No matter what language you learn, it is almost always beneficial to learn other programming languages too, at least at a basic level. I have literally never known a good programmer who knew only one language.

I know about five of them well, and another five superficially (I played with Python for a day a few years ago). An advantage is that each has its own insights, paradigms, clever way of solving some problems.

A disadvantage of programming in other languages is that it becomes easier to mix up. I have written statements in my julia program like

(x>0) or x=0
(x>0) || x=0
function f(const int x); x+2; end

and then stared at the error messages for a while before it dawned on me that I was no longer in Kansas. these three may also be wishful thinking, with me muttering under my breadth about why julia had to chose yet another somewhat different syntax from my bread-and-butter languages.

A big julia appeal for me is that I hope it will be able to cover most of my C, C++, R, and perl needs. (I went perl before python took over perl’s bread and butter.) if I had lots of time (no) and if I did not tend towards mixing up metaphors (no), I would play around more with python. I am recommending Python to my students.

/iaw

[BLI]

I tried to use PGFPlotsX (?) as a backend for Plots about a year ago, without success. Maybe I did something wrong…

OK – PyPlot supports LaTeX syntax for text (although in IJulia, the result certainly does not look like Computer Modern font…). Are there other backends that support LaTeX now?

[BLI]

I have saved figures to PDF, and the result is very nice. Currently, I tend to save figures as SVG files, which also leads to very nice result. (I don’t know the relative advantage of PDF vs. SVG, though.)