IJulia and Pyplot backend problem with old Python version

[Freya_the_Goddess]

Hi all,

I have upgraded my Python to 3.10 just to be able to use Pyplot backend (do not know if it is worth the trouble, perhaps other backend is better)

I want to plot this:

using Plots; pyplot();

xc = [0,0,1,1]
yc = [0,1,1,0]
zc = [1,1,2,2];

plot(xc,yc,zc,st=:surface,camera=(-30,30))

Then error occurs:
Warning: You are using Matplotlib 3.3.4, which is no longer
│ officialy supported by the Plots community. To ensure smooth Plots.jl
│ integration update your Matplotlib library to a version >= 3.4.0
**│ **
│ If you have used Conda.jl to install PyPlot (default installation),
│ upgrade your matplotlib via Conda.jl and rebuild the PyPlot.
**│ **
│ If you are not sure, here are the default instructions:
**│ **
│ In Julia REPL:
│ ** **│ import Pkg;** **│ Pkg.add("Conda")** **│ import Conda** **│ Conda.update()** **│ Pkg.build("PyPlot")** **│
**│ **
└ @ Plots /home/browni/.julia/packages/Plots/SkUg1/src/backends/pyplot.jl:29

I want to change the kernel of Jupyter Notebook to the newest Python 3.10 so I can use pyplot() backend.

This also makes the notebook for Python kernel that used to work with PyplotJS for Python is not working now as well.

Just how to fix this Python kernel ? I have the newest matplotlib now when I check at terminal.

Different error when I run the code again:

[nilshg]

The Python version used by IJulia is likely a miniconda downloaded automatically when you installed IJulia, not your system Python. You can either re-build IJulia with your system Python or update the matplotlib version used by the Python used by IJulia (as the warning suggests).

If you ask me neither of these is worth the trouble unless you need a specific feature of the PyPlot backend which isn’t supported by GR.

Also the warning is just that, a warning, are you getting actual errors which prevent you from plotting?

[Paul_Soderlind]

Not an expert, but this notebook has some simple instructions for “Installing Jupyter/JupyterLab - When You Already Have Python Installed” (close to the top of the notebook).

Also, consider checking with
using PyPlot;PyPlot.version and run(`$(ENV["JUPYTER"]) --version`)

[Freya_the_Goddess]

It is a warning and unable to show the plot, Pyplot can rotate the plot that is why I need it.

I follow this :

│ In Julia REPL:
│ ```
│ import Pkg;
│ Pkg.add("Conda")
│ import Conda
│ Conda.update()
│ Pkg.build("PyPlot")

I get this:

Error building `PyCall`: 
<string>:1: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives
<string>:1: DeprecationWarning: The distutils.sysconfig module is deprecated, use sysconfig instead
<string>:1: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives
<string>:1: DeprecationWarning: The distutils.sysconfig module is deprecated, use sysconfig instead
ERROR: LoadError: Couldn't find libpython; check your PYTHON environment variable.

The python executable we tried was /usr/bin/python3 (= version 3.10).
Re-building with
    ENV["PYCALL_DEBUG_BUILD"] = "yes"
may provide extra information for why it failed.

Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:33
 [2] find_libpython(python::SubString{String}; _dlopen::typeof(Base.Libc.Libdl.dlopen))
   @ Main ~/.julia/packages/PyCall/7a7w0/deps/buildutils.jl:72
 [3] find_libpython(python::SubString{String})
   @ Main ~/.julia/packages/PyCall/7a7w0/deps/buildutils.jl:41
 [4] top-level scope
   @ ~/.julia/packages/PyCall/7a7w0/deps/build.jl:82
 [5] include(fname::String)
   @ Base.MainInclude ./client.jl:451
 [6] top-level scope
   @ none:5
in expression starting at /home/browni/.julia/packages/PyCall/7a7w0/deps/build.jl:43
Stacktrace:
  [1] pkgerror(msg::String)
    @ Pkg.Types ~/julia-1.7.3/share/julia/stdlib/v1.7/Pkg/src/Types.jl:68
  [2] (::Pkg.Operations.var"#62#67"{Bool, Pkg.Types.Context, String, Pkg.Types.PackageSpec})()
    @ Pkg.Operations ~/julia-1.7.3/share/julia/stdlib/v1.7/Pkg/src/Operations.jl:962
  [3] withenv(::Pkg.Operations.var"#62#67"{Bool, Pkg.Types.Context, String, Pkg.Types.PackageSpec}, ::Pair{String, String}, ::Vararg{Pair{String}})
    @ Base ./env.jl:172
  [4] (::Pkg.Operations.var"#99#103"{String, Bool, Bool, Bool, Pkg.Operations.var"#62#67"{Bool, Pkg.Types.Context, String, Pkg.Types.PackageSpec}, Pkg.Types.PackageSpec})()
    @ Pkg.Operations ~/julia-1.7.3/share/julia/stdlib/v1.7/Pkg/src/Operations.jl:1506
  [5] with_temp_env(fn::Pkg.Operations.var"#99#103"{String, Bool, Bool, Bool, Pkg.Operations.var"#62#67"{Bool, Pkg.Types.Context, String, Pkg.Types.PackageSpec}, Pkg.Types.PackageSpec}, temp_env::String)
    @ Pkg.Operations ~/julia-1.7.3/share/julia/stdlib/v1.7/Pkg/src/Operations.jl:1390
  [6] (::Pkg.Operations.var"#98#102"{Bool, Bool, Bool, Pkg.Operations.var"#62#67"{Bool, Pkg.Types.Context, String, Pkg.Types.PackageSpec}, Pkg.Types.Context, Pkg.Types.PackageSpec, String, Pkg.Types.Project, String})(tmp::String)
    @ Pkg.Operations ~/julia-1.7.3/share/julia/stdlib/v1.7/Pkg/src/Operations.jl:1469
  [7] mktempdir(fn::Pkg.Operations.var"#98#102"{Bool, Bool, Bool, Pkg.Operations.var"#62#67"{Bool, Pkg.Types.Context, String, Pkg.Types.PackageSpec}, Pkg.Types.Context, Pkg.Types.PackageSpec, String, Pkg.Types.Project, String}, parent::String; prefix::String)
    @ Base.Filesystem ./file.jl:750
  [8] mktempdir(fn::Function, parent::String) (repeats 2 times)
    @ Base.Filesystem ./file.jl:748
  [9] sandbox(fn::Function, ctx::Pkg.Types.Context, target::Pkg.Types.PackageSpec, target_path::String, sandbox_path::String, sandbox_project_override::Pkg.Types.Project; force_latest_compatible_version::Bool, allow_earlier_backwards_compatible_versions::Bool, allow_reresolve::Bool)
    @ Pkg.Operations ~/julia-1.7.3/share/julia/stdlib/v1.7/Pkg/src/Operations.jl:1435
 [10] sandbox(fn::Function, ctx::Pkg.Types.Context, target::Pkg.Types.PackageSpec, target_path::String, sandbox_path::String, sandbox_project_override::Pkg.Types.Project)
    @ Pkg.Operations ~/julia-1.7.3/share/julia/stdlib/v1.7/Pkg/src/Operations.jl:1432
 [11] build_versions(ctx::Pkg.Types.Context, uuids::Set{Base.UUID}; verbose::Bool)
    @ Pkg.Operations ~/julia-1.7.3/share/julia/stdlib/v1.7/Pkg/src/Operations.jl:943
 [12] build(ctx::Pkg.Types.Context, uuids::Set{Base.UUID}, verbose::Bool)
    @ Pkg.Operations ~/julia-1.7.3/share/julia/stdlib/v1.7/Pkg/src/Operations.jl:822
 [13] build(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec}; verbose::Bool, kwargs::Base.Pairs{Symbol, Base.TTY, Tuple{Symbol}, NamedTuple{(:io,), Tuple{Base.TTY}}})
    @ Pkg.API ~/julia-1.7.3/share/julia/stdlib/v1.7/Pkg/src/API.jl:992
 [14] build(pkgs::Vector{Pkg.Types.PackageSpec}; io::Base.TTY, kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ Pkg.API ~/julia-1.7.3/share/julia/stdlib/v1.7/Pkg/src/API.jl:149
 [15] build(pkgs::Vector{Pkg.Types.PackageSpec})
    @ Pkg.API ~/julia-1.7.3/share/julia/stdlib/v1.7/Pkg/src/API.jl:144
 [16] #build#99
    @ ~/julia-1.7.3/share/julia/stdlib/v1.7/Pkg/src/API.jl:142 [inlined]
 [17] build
    @ ~/julia-1.7.3/share/julia/stdlib/v1.7/Pkg/src/API.jl:142 [inlined]
 [18] #build#98
    @ ~/julia-1.7.3/share/julia/stdlib/v1.7/Pkg/src/API.jl:141 [inlined]
 [19] build(pkg::String)
    @ Pkg.API ~/julia-1.7.3/share/julia/stdlib/v1.7/Pkg/src/API.jl:141
 [20] top-level scope
    @ REPL[4]:1

[Freya_the_Goddess]

I am reading it, great tutorial…

I think my problem is very complicated .

[Freya_the_Goddess]

PyCall and GMT failed to precompiled

[Freya_the_Goddess]

So after a lot of trial and error and help from all of you here.

I realize upgrading Python the main system is useless, sorry Python!!

So here what I do, cd to an empty directory for a new environment then:

julia --project="."

Then at REPL:

julia> ENV["PYTHON"]="/usr/bin/python3.6"
(to get back to the original python installed at the system)

julia> ]
pkg> add PyCall

julia> using Pkg
julia> Pkg.build("PyCall")

julia> ]
pkg> add PyPlot
julia> Pkg.build("PyPlot")

julia> using IJulia
julia> notebook()

Then at the notebook (the python kernel works again to plot, but the matplotlib problem still persists in Julia kernel):

The right question is how to install new matplotlib in conda in Julia?
so I can do pyplot() in IJulia

[Freya_the_Goddess]

After a bit trial and error,

I open a notebook with Python kernel, and check turns out if I run pip list it will point to conda inside .julia

Then I install matplotlib in a new notebook by :

pip install matplotlib
pip show matplotlib

Go back to Julia kernel notebook to plot an example with pyplot() backend (still the same):

the same as I use terminal for REPL when I call pyplot()

I will try to find a way on how to install matplotlib in conda in Julia then.

using Conda, does not really work to install matplotlib, it installed python 3.10 but then this error occurs:

CondaVerificationError: The package for python located at /home/browni/.julia/conda/3/pkgs/python-3.10.5-h582c2e5_0_cpython
appears to be corrupted. The path ‘lib/python3.10/unittest/pycache/loader.cpython-310.pyc’
specified in the package manifest cannot be found.

[Freya_the_Goddess]

https://github.com/JuliaPy/Conda.jl/issues/222

[Freya_the_Goddess]

I try this:

I have tried this codes in Julia REPL:

julia> using Conda
julia> Conda.add("matplotlib", :pyplot_env)

It created a new conda environment at this location: /home/browni/.julia/conda/3/envs/pyplot_env

At the terminal it said:
To activate this environment, use

  $ conda activate /home/browni/.julia/conda/3/envs/pyplot_env

I activate this conda environment at OS terminal I type:

source /home/browni/.julia/conda/3/bin/activate 
conda activate pyplot_env

It activates pyplot_env in terminal.

Then, cd to the empty directory for clean environment for Julia then type:

julia --project="."

julia> import Pkg;
julia> Pkg.add("Conda")
julia> import Conda
julia> Conda.update()
julia> Pkg.build("PyPlot")

Still cannot make the Pyplot() backend works… Try a lot of ways, but will try again. If anyone able to share their experience in same issue feel free to help me here.

[Freya_the_Goddess]

Case closed / Solution obtained:

I create a new folder pyplotenv, and cd to there then type this from terminal:

julia --project="."

import Conda
Conda.update()

add PyPlot

Then run the plotting:

using Plots
pyplot()

xc = [0,0,1,1]
yc = [0,1,1,0]
zc = [1,1,2,2];

plot(xc,yc,zc,st=:surface,camera=(-30,30))

in Julia REPL and IJulia

Now it works. Hope this can help others who face issue with matplotlib and pyplot backend.

[Paul_Soderlind]

In short, what was the problem? A “dirty” old installation?

[Freya_the_Goddess]

The problem is my environment is looking or using the wrong Python.

Thus, I need to set Python Env to Python in Conda.jl, then set the Conda environment variable so I can use new matplotlib for pyplot() backend:

julia> ENV["PYTHON"] = "/home/browni/.julia/conda/3/envs/pyplot_env/bin/python3"
julia> ENV["CONDA_JL_HOME"] = "/home/browni/.julia/conda/3/envs/pyplot_env"