PyPlot error on Julia 1.0.1

Hi all,

I am having some trouble getting PyPlot to work with Julia 1.0.1. Starting with a fresh install, running from command line, here’s what I got (sorry it’s a little long):

   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.0.1 (2018-09-29)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

┌ Warning: Terminal not fully functional
└ @ Base client.jl:342
julia> Pkg.status()
ERROR: UndefVarError: Pkg not defined
Stacktrace:
 [1] top-level scope at none:0

julia> using Pkg

julia> Pkg.status()
    Status `~/.julia/environments/v1.0/Project.toml`

julia> Pkg.add("PyPlot")
   Cloning default registries into /Users/kkylin/.julia/registries
   Cloning registry General from "https://github.com/JuliaRegistries/General.git"
  Updating registry at `~/.julia/registries/General`
  Updating git-repo `https://github.com/JuliaRegistries/General.git`
 Resolving package versions...
 Installed Reexport ────────── v0.2.0
 Installed VersionParsing ──── v1.1.3
 Installed FixedPointNumbers ─ v0.5.3
 Installed JSON ────────────── v0.19.0
 Installed LaTeXStrings ────── v1.0.3
 Installed PyPlot ──────────── v2.6.3
 Installed PyCall ──────────── v1.18.5
 Installed Conda ───────────── v1.1.1
 Installed MacroTools ──────── v0.4.4
 Installed Compat ──────────── v1.3.0
 Installed Colors ──────────── v0.9.4
 Installed ColorTypes ──────── v0.7.5
  Updating `~/.julia/environments/v1.0/Project.toml`
  [d330b81b] + PyPlot v2.6.3
  Updating `~/.julia/environments/v1.0/Manifest.toml`
  [3da002f7] + ColorTypes v0.7.5
  [5ae59095] + Colors v0.9.4
  [34da2185] + Compat v1.3.0
  [8f4d0f93] + Conda v1.1.1
  [53c48c17] + FixedPointNumbers v0.5.3
  [682c06a0] + JSON v0.19.0
  [b964fa9f] + LaTeXStrings v1.0.3
  [1914dd2f] + MacroTools v0.4.4
  [438e738f] + PyCall v1.18.5
  [d330b81b] + PyPlot v2.6.3
  [189a3867] + Reexport v0.2.0
  [81def892] + VersionParsing v1.1.3
  [2a0f44e3] + Base64 
  [ade2ca70] + Dates 
  [8bb1440f] + DelimitedFiles 
  [8ba89e20] + Distributed 
  [b77e0a4c] + InteractiveUtils 
  [76f85450] + LibGit2 
  [8f399da3] + Libdl 
  [37e2e46d] + LinearAlgebra 
  [56ddb016] + Logging 
  [d6f4376e] + Markdown 
  [a63ad114] + Mmap 
  [44cfe95a] + Pkg 
  [de0858da] + Printf 
  [3fa0cd96] + REPL 
  [9a3f8284] + Random 
  [ea8e919c] + SHA 
  [9e88b42a] + Serialization 
  [1a1011a3] + SharedArrays 
  [6462fe0b] + Sockets 
  [2f01184e] + SparseArrays 
  [10745b16] + Statistics 
  [8dfed614] + Test 
  [cf7118a7] + UUIDs 
  [4ec0a83e] + Unicode 
  Building Conda ─→ `~/.julia/packages/Conda/uQitS/deps/build.log`
  Building PyCall → `~/.julia/packages/PyCall/0jMpb/deps/build.log`

julia> using PyPlot
[ Info: Precompiling PyPlot [d330b81b-6aea-500a-939a-2ce795aea3ee]
[ Info: Installing matplotlib via the Conda matplotlib package...
[ Info: Running `conda install -y matplotlib` in root environment
Solving environment: done

## Package Plan ##

  environment location: /Users/kkylin/.julia/conda/3

  added / updated specs: 
    - matplotlib


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    python-dateutil-2.7.3      |           py37_0         260 KB
    pytz-2018.5                |           py37_0         231 KB
    matplotlib-3.0.0           |   py37h54f8f79_0         6.8 MB
    pyparsing-2.2.2            |           py37_0          97 KB
    kiwisolver-1.0.1           |   py37h0a44026_0          56 KB
    freetype-2.9.1             |       hb4e5f40_0         864 KB
    libpng-1.6.35              |       ha441bb4_0         318 KB
    cycler-0.10.0              |           py37_0          14 KB
    tornado-5.1.1              |   py37h1de35cc_0         661 KB
    ------------------------------------------------------------
                                           Total:         9.3 MB

The following NEW packages will be INSTALLED:

    cycler:          0.10.0-py37_0       
    freetype:        2.9.1-hb4e5f40_0    
    kiwisolver:      1.0.1-py37h0a44026_0
    libpng:          1.6.35-ha441bb4_0   
    matplotlib:      3.0.0-py37h54f8f79_0
    pyparsing:       2.2.2-py37_0        
    python-dateutil: 2.7.3-py37_0        
    pytz:            2018.5-py37_0       
    tornado:         5.1.1-py37h1de35cc_0


Downloading and Extracting Packages
python-dateutil-2.7. | 260 KB    | ################################################################################################################ | 100% 
pytz-2018.5          | 231 KB    | ################################################################################################################ | 100% 
matplotlib-3.0.0     | 6.8 MB    | ################################################################################################################ | 100% 
pyparsing-2.2.2      | 97 KB     | ################################################################################################################ | 100% 
kiwisolver-1.0.1     | 56 KB     | ################################################################################################################ | 100% 
freetype-2.9.1       | 864 KB    | ################################################################################################################ | 100% 
libpng-1.6.35        | 318 KB    | ################################################################################################################ | 100% 
cycler-0.10.0        | 14 KB     | ################################################################################################################ | 100% 
tornado-5.1.1        | 661 KB    | ################################################################################################################ | 100% 
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
ERROR: InitError: PyError (PyImport_ImportModule) <class 'RuntimeError'>
RuntimeError("The current Numpy installation ('/Users/kkylin/.julia/conda/3/lib/python3.7/site-packages/numpy/__init__.py') fails to pass simple sanity checks. This can be caused for example by incorrect BLAS library being linked in.")
  File "/Users/kkylin/.julia/conda/3/lib/python3.7/site-packages/matplotlib/__init__.py", line 141, in <module>
    from . import cbook, rcsetup
  File "/Users/kkylin/.julia/conda/3/lib/python3.7/site-packages/matplotlib/cbook/__init__.py", line 33, in <module>
    import numpy as np
  File "/Users/kkylin/.julia/conda/3/lib/python3.7/site-packages/numpy/__init__.py", line 227, in <module>
    _sanity_check()
  File "/Users/kkylin/.julia/conda/3/lib/python3.7/site-packages/numpy/__init__.py", line 225, in _sanity_check
    raise RuntimeError(msg.format(__file__))

Stacktrace:
 [1] pyimport(::String) at /Users/kkylin/.julia/packages/PyCall/0jMpb/src/PyCall.jl:486
 [2] macro expansion at ./logging.jl:309 [inlined]
 [3] pyimport_conda(::String, ::String, ::String) at /Users/kkylin/.julia/packages/PyCall/0jMpb/src/PyCall.jl:653
 [4] pyimport_conda at /Users/kkylin/.julia/packages/PyCall/0jMpb/src/PyCall.jl:649 [inlined]
 [5] __init__() at /Users/kkylin/.julia/packages/PyPlot/fZuOQ/src/init.jl:174
 [6] _include_from_serialized(::String, ::Array{Any,1}) at ./loading.jl:630
 [7] _require_from_serialized(::String) at ./loading.jl:681
 [8] _require(::Base.PkgId) at ./logging.jl:317
 [9] require(::Base.PkgId) at ./loading.jl:855
 [10] macro expansion at ./logging.jl:311 [inlined]
 [11] require(::Module, ::Symbol) at ./loading.jl:837
during initialization of module PyPlot

As the error suggests, this seems to be a problem involving libopenblas being linked somewhere. But I couldn’t quite figure out a fix. Any pointers would be greatly appreciated!

KL

It’s useful to check if it works with Python alone first:

using Pkg
Pkg.add("PyCall")
using PyCall
# Check if it's importable by Python
run(`$(PyCall.pyprogramname) -c "import numpy"`)
run(`$(PyCall.pyprogramname) -c "import matplotlib"`)
# Check if it's importable by PyCall
pyimport("numpy")
pyimport("matplotlib")
# Finally try PyPlot
using PyPlot

Note: the error is coming from

https://github.com/numpy/numpy/blob/8aa121415760cc6839a546c3f84e238d1dfa1aa6/numpy/__init__.py#L193-L214

which mentions

Thanks for the suggestion. Here’s what I got:

julia> using Pkg

julia> using PyCall

julia> run(`$(PyCall.pyprogramname) -c "import numpy"`)
Process(`/Users/kkylin/.julia/conda/3/bin/python -c 'import numpy'`, ProcessExited(0))

julia> run(`$(PyCall.pyprogramname) -c "import matplotlib"`)
Process(`/Users/kkylin/.julia/conda/3/bin/python -c 'import matplotlib'`, ProcessExited(0))

julia> pyimport("numpy")
ERROR: PyError (PyImport_ImportModule) <class 'RuntimeError'>
RuntimeError("The current Numpy installation ('/Users/kkylin/.julia/conda/3/lib/python3.7/site-packages/numpy/__init__.py') fails to pass simple sanity checks. This can be caused for example by incorrect BLAS library being linked in.")
  File "/Users/kkylin/.julia/conda/3/lib/python3.7/site-packages/numpy/__init__.py", line 227, in <module>
    _sanity_check()
  File "/Users/kkylin/.julia/conda/3/lib/python3.7/site-packages/numpy/__init__.py", line 225, in _sanity_check
    raise RuntimeError(msg.format(__file__))

Stacktrace:
 [1] pyimport(::String) at /Users/kkylin/.julia/packages/PyCall/0jMpb/src/PyCall.jl:486
 [2] top-level scope at none:0

julia> pyimport("matplotlib")
ERROR: PyError (PyImport_ImportModule) <class 'RuntimeError'>
RuntimeError("The current Numpy installation ('/Users/kkylin/.julia/conda/3/lib/python3.7/site-packages/numpy/__init__.py') fails to pass simple sanity checks. This can be caused for example by incorrect BLAS library being linked in.")
  File "/Users/kkylin/.julia/conda/3/lib/python3.7/site-packages/matplotlib/__init__.py", line 126, in <module>
    from . import cbook
  File "/Users/kkylin/.julia/conda/3/lib/python3.7/site-packages/matplotlib/cbook/__init__.py", line 34, in <module>
    import numpy as np
  File "/Users/kkylin/.julia/conda/3/lib/python3.7/site-packages/numpy/__init__.py", line 227, in <module>
    _sanity_check()
  File "/Users/kkylin/.julia/conda/3/lib/python3.7/site-packages/numpy/__init__.py", line 225, in _sanity_check
    raise RuntimeError(msg.format(__file__))

Stacktrace:
 [1] pyimport(::String) at /Users/kkylin/.julia/packages/PyCall/0jMpb/src/PyCall.jl:486
 [2] top-level scope at none:0


So indeed, Julia cannot import numpy. However, the numpy installation seems to be fine when called from within python:

julia> run(`/Users/kkylin/.julia/conda/3/bin/python`)
Python 3.7.0 (default, Jun 28 2018, 07:39:16) 
[Clang 4.0.1 (tags/RELEASE_401/final)] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
>>> a = np.arange(15).reshape(3, 5)
>>> a
array([[ 0,  1,  2,  3,  4],
       [ 5,  6,  7,  8,  9],
       [10, 11, 12, 13, 14]])
>>> a.shape
(3, 5)
>>> a.ndim
2
>>> 
Process(`/Users/kkylin/.julia/conda/3/bin/python`, ProcessExited(0))

julia> 

(I do have other pythons on my system in various places, but this looks like the one Julia is using.)

I’ll keep looking.

Thanks,
KL

Alright, looks like the culprit is having multiple python distributions. In particular, I have Anaconda installed (separately, manually) elsewhere in my home directory, and that seemed to interfere with the version in .julia/conda. When I set PyCall to use Anaconda, the problems went away.

For anyone else having similar issues, try:

ENV["PYTHON"] = "/Users/foo/anaconda/bin/python"  ## put the path to your python program here
Pkg.build("PyCall")

And that may (hopefully!) take care of it.

KL

Actually, the test for matplotlib should be something like

run(`/Users/kkylin/.julia/conda/3/bin/python -c "from matplotlib.pyplot import plot, show; plot([1,2,3]); show()"`)

since just importing matplotlib won’t activate the GUI. Or maybe use Python REPL as you did.

julia> run(`/Users/kkylin/.julia/conda/3/bin/python`)
Python 3.7.0 ...
>>> from matplotlib.pyplot import plot, show
>>> plot([1,2,3])
>>> show()

Yes, that’s correct. In general, whatever the value of PyCall.pyprogramname is the Python used by Julia.

If it works with ~/anaconda/bin/python I guess there is a combination of numpy, matplotlib (+ other Python packages) and PyCall that does not work on macOS. Judging from the value of your PyCall.pyprogramname, I guess you installed/updated Conda.jl after installing/updating ~/anaconda? If that’s the case, you may want to be careful updating ~/anaconda until the bug is solved.

If it works with ~/anaconda/bin/python I guess
there is a combination of numpy, matplotlib (+ other
Python packages) and PyCall that does not work on
macOS. Judging from the value of your
PyCall.pyprogramname, I guess you
installed/updated Conda.jl after installing/updating
~/anaconda? If that’s the case, you may want to
be careful updating ~/anaconda until the bug is
solved.

Yes, I think you are right. My installation with
regard to python is a bit of a mess. I do need to
clean it up.

Thanks for the help!

KL