# Can not install PyCall and PyPlot on HPC cluster

**URL:** <https://discourse.julialang.org/t/can-not-install-pycall-and-pyplot-on-hpc-cluster/68466>\
**Category:** General Usage\
**Tags:** question\
**Created:** [September 20, 2021, 2:19pm UTC](https://discourse.julialang.org/t/can-not-install-pycall-and-pyplot-on-hpc-cluster/68466 "2021-09-20T14:19:26Z")\
**Posts on this page:** 17\
**Page:** 1

<div class="post-metadata">

**Author:** ![fksayaci](https://avatars.discourse-cdn.com/v4/letter/f/b77776/32.png) [@fksayaci](https://discourse.julialang.org/u/fksayaci)\
**Post date:** [September 20, 2021, 2:19pm UTC](https://discourse.julialang.org/t/can-not-install-pycall-and-pyplot-on-hpc-cluster/68466/1 "2021-09-20T14:19:26Z")

</div>

Hi All,

I am using a high-performance computing cluster to solve my optimization problem. After I get the results, I need to plot results on bar plots, line plots, and stack graphs. I am doing these successfully on my local machine, but it gives error on the cluster in `using` stage.

When I have the following

`using JuMP, PyPlot, DataFrames, CSV, Gurobi, Missings, PyCall, Statistics, TimerOutputs`

as a first line, it gives error below:

 ![image](https://global.discourse-cdn.com/julialang/original/3X/b/3/b3dc63b500c251d82f7fe7df1d5768fbfebadf98.png)

So, it tells me that I do not have the required installation. When I have the following as first lines,

```julia
using Pkg
Pkg.add("PyCall")
using PyCall
Pkg.add("PyPlot")
using PyPlot

using JuMP, DataFrames, CSV, Gurobi, Missings, Statistics, TimerOutputs

```

it gives the following error:

 ![using Pkg, Pkg add and using PyCall and PyPlot, then using everything rest](https://global.discourse-cdn.com/julialang/original/3X/8/7/8752691f8fdade09f6e30d8caf7b171fdddfe352.png)

So, there is a problem in Line 6, which is `Pkg.add("PyCall")`.

Can you please help me with this?

Kind regards,  
Fikri Kucuksayacigil

---

<div class="post-metadata">

**Author:** ![simonbyrne](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/simonbyrne/32/19_2.png) [@simonbyrne](https://discourse.julialang.org/u/simonbyrne)\
**Post date:** [September 20, 2021, 2:34pm UTC](https://discourse.julialang.org/t/can-not-install-pycall-and-pyplot-on-hpc-cluster/68466/2 "2021-09-20T14:34:21Z")

</div>

It could be that the cluster worker node doesn’t have direct internet access? You may need to download and configure the packages on the login node beforehand.

---

<div class="post-metadata">

**Author:** ![simonbyrne](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/simonbyrne/32/19_2.png) [@simonbyrne](https://discourse.julialang.org/u/simonbyrne)\
**Post date:** [September 20, 2021, 2:36pm UTC](https://discourse.julialang.org/t/can-not-install-pycall-and-pyplot-on-hpc-cluster/68466/3 "2021-09-20T14:36:01Z")

</div>

This could also have been an error on the package server side: I just saw a similar error in one of my CI jobs.

---

<div class="post-metadata">

**Author:** ![fksayaci](https://avatars.discourse-cdn.com/v4/letter/f/b77776/32.png) [@fksayaci](https://discourse.julialang.org/u/fksayaci)\
**Post date:** [September 20, 2021, 3:36pm UTC](https://discourse.julialang.org/t/can-not-install-pycall-and-pyplot-on-hpc-cluster/68466/4 "2021-09-20T15:36:55Z")

</div>

Thank you for your message. But, I think the problem is not internet connection because I was able to run the same code on the cluster without PyCall and PyPlot before. Recently, I started feeling that I need figures. That is why I integrated PyCall and PyPlot. Then, this error happens.

What do you suggest me? Should I try to install a simple package to see if the problem is about internet connection?

---

<div class="post-metadata">

**Author:** ![johnh](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/johnh/32/3615_2.png) [@johnh](https://discourse.julialang.org/u/johnh)\
**Post date:** [September 20, 2021, 5:04pm UTC](https://discourse.julialang.org/t/can-not-install-pycall-and-pyplot-on-hpc-cluster/68466/5 "2021-09-20T17:04:19Z")

</div>

What @simonbyrne is saying is that the compute nodes on the cluster may not have internet access.  
Try logging into the login node and running Julia in the REPL then adding the packages you need.

---

<div class="post-metadata">

**Author:** ![simonbyrne](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/simonbyrne/32/19_2.png) [@simonbyrne](https://discourse.julialang.org/u/simonbyrne)\
**Post date:** [September 20, 2021, 5:10pm UTC](https://discourse.julialang.org/t/can-not-install-pycall-and-pyplot-on-hpc-cluster/68466/6 "2021-09-20T17:10:46Z")

</div>

There was also an issue with the package server today, so that could have been the issue. Try it again?

---

<div class="post-metadata">

**Author:** ![aramirezreyes](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/aramirezreyes/32/42573_2.png) [@aramirezreyes](https://discourse.julialang.org/u/aramirezreyes)\
**Post date:** [September 20, 2021, 5:12pm UTC](https://discourse.julialang.org/t/can-not-install-pycall-and-pyplot-on-hpc-cluster/68466/7 "2021-09-20T17:12:56Z")

</div>

I think you have installed PyCall correctly. It is only that it is trying to call the python installation that is by default in the cluster, and that one doesn’t have matplotlib installed.

The easiest would be to read the third option given in your first error message (the paragraph starting with Another alternative…). For this you will probably need internet access, which, as Simon says may not be available in the compute nodes.

So follow those instructions in the login node to get your environment correctly configured.

---

<div class="post-metadata">

**Author:** ![natgeo-wong](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/natgeo-wong/32/10965_2.png) [@natgeo-wong](https://discourse.julialang.org/u/natgeo-wong)\
**Post date:** [September 20, 2021, 5:41pm UTC](https://discourse.julialang.org/t/can-not-install-pycall-and-pyplot-on-hpc-cluster/68466/8 "2021-09-20T17:41:13Z")

</div>

I used to run into this error all the time on my cluster. So what I did was

```julia
ENV["PYTHON"] = (path of Python executable for Julia's miniconda)
pkg> build PyCall

```

This would force the python that I used to be the one from Julia.

---

<div class="post-metadata">

**Author:** ![fksayaci](https://avatars.discourse-cdn.com/v4/letter/f/b77776/32.png) [@fksayaci](https://discourse.julialang.org/u/fksayaci)\
**Post date:** [September 20, 2021, 5:54pm UTC](https://discourse.julialang.org/t/can-not-install-pycall-and-pyplot-on-hpc-cluster/68466/9 "2021-09-20T17:54:59Z")

</div>

Sorry for this dumb question, but how do I start Julia session on cluster terminal? When I type `julia`, it says `command not found`. When I type `python`, it starts a Python session.

---

<div class="post-metadata">

**Author:** ![mkitti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkitti/32/12459_2.png) [@mkitti](https://discourse.julialang.org/u/mkitti)\
**Post date:** [September 20, 2021, 6:32pm UTC](https://discourse.julialang.org/t/can-not-install-pycall-and-pyplot-on-hpc-cluster/68466/10 "2021-09-20T18:32:18Z")

</div>

You should probably have a conversion with your HPC admin. You appear to trying to use the system python, which is probably not the latest python.

On some HPCs you need to do something like `module load python julia` in order to load python and julia into your shell environment.

Since you can start Python, what happens when you type “import matplotlib” there?

---

<div class="post-metadata">

**Author:** ![fksayaci](https://avatars.discourse-cdn.com/v4/letter/f/b77776/32.png) [@fksayaci](https://discourse.julialang.org/u/fksayaci)\
**Post date:** [September 20, 2021, 6:47pm UTC](https://discourse.julialang.org/t/can-not-install-pycall-and-pyplot-on-hpc-cluster/68466/11 "2021-09-20T18:47:17Z")

</div>

It says `No module named matplotlib`

I am currently trying to start a Julia session in terminal and will do `Pkg.add("PyCall")` and `Pkg.add("PyPlot")` in the terminal. I am trying to make Julia callable through terminal. It seems that I have to add an environment variable pointing to `julia/bin`, but I have not got yet where julia was installed (what is the path?).

---

<div class="post-metadata">

**Author:** ![mkitti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkitti/32/12459_2.png) [@mkitti](https://discourse.julialang.org/u/mkitti)\
**Post date:** [September 20, 2021, 7:23pm UTC](https://discourse.julialang.org/t/can-not-install-pycall-and-pyplot-on-hpc-cluster/68466/12 "2021-09-20T19:23:12Z")

</div>

> [@fksayaci](#):
>
> It says `No module named matplotlib`

This means that your Python environment does not have matplitlib installed. This is a prerequisite.

> [@fksayaci](#):
>
> I am trying to make Julia callable through terminal. It seems that I have to add an environment variable pointing to `julia/bin` , but I have not got yet where julia was installed (what is the path?).

This is specific to your HPC. You need to talk to whomever installed Julia on your HPC system, unless you installed Julia yourself.

Can you tell us which HPC you are using? If it is public, I might be able to help you find public information about it.

---

<div class="post-metadata">

**Author:** ![fksayaci](https://avatars.discourse-cdn.com/v4/letter/f/b77776/32.png) [@fksayaci](https://discourse.julialang.org/u/fksayaci)\
**Post date:** [September 20, 2021, 7:54pm UTC](https://discourse.julialang.org/t/can-not-install-pycall-and-pyplot-on-hpc-cluster/68466/13 "2021-09-20T19:54:20Z")

</div>

It seems that I progressed a little.

First of all, my terminal windows (in my local machine and cluster) were not able to start Julia. I figured out how to make Julia initiated through terminal windows on Mac and Linux. I now can use Julia in terminal at the cluster.

When I typed `Pkg.status()`, I see

 ![Screen Shot 2021-09-20 at 12.39.02 PM](https://global.discourse-cdn.com/julialang/original/3X/5/5/551362af66130cae3b4356e9fe54d94586221363.png)

So, it seems that packages are already installed. When I type `using PyCall`, there is no error. When I type `using PyPlot`, it gives the following error:

 ![Screen Shot 2021-09-20 at 12.50.50 PM](https://global.discourse-cdn.com/julialang/original/3X/5/4/54be91d05a301194ba32d94fb576abff50dc2632.png)

I will follow the suggestions given in this error.

By the way @mkitti, I am using University of California San Diego’s HPC for my work. I am pretty new to cluster computing. So, it takes some time for me to figure out things.

---

<div class="post-metadata">

**Author:** ![fksayaci](https://avatars.discourse-cdn.com/v4/letter/f/b77776/32.png) [@fksayaci](https://discourse.julialang.org/u/fksayaci)\
**Post date:** [September 21, 2021, 4:32am UTC](https://discourse.julialang.org/t/can-not-install-pycall-and-pyplot-on-hpc-cluster/68466/14 "2021-09-21T04:32:23Z")

</div>

I think I did it. I just typed

```julia
ENV["PYTHON"]=""
Pkg.build("PyCall")

```

and then restarted Julia, as described in the error message. I am now able to do `using PyPlot` without any error message.

---

<div class="post-metadata">

**Author:** ![mkitti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkitti/32/12459_2.png) [@mkitti](https://discourse.julialang.org/u/mkitti)\
**Post date:** [September 21, 2021, 9:32pm UTC](https://discourse.julialang.org/t/can-not-install-pycall-and-pyplot-on-hpc-cluster/68466/15 "2021-09-21T21:32:58Z")

</div>

Essentially what this did is create a distinct Python install using miniconda for Julia. Before it was trying to use the operating system’s Python install. This works, but I am not sure if it is the best use of HPC resources.

What I suspect is there is probably is an existing Python “module” somewhere on the system for you so you don’t have to have a full copy in your home directory. On the login node, if you type “module avail” do you see a list of software that includes Python?

I have not found any UCSD specific instructions. Most HPCs I have used work like this:  
[https://www.glue.umd.edu/hpcc/help/software/python.html](https://www.glue.umd.edu/hpcc/help/software/python.html)

---

<div class="post-metadata">

**Author:** ![fksayaci](https://avatars.discourse-cdn.com/v4/letter/f/b77776/32.png) [@fksayaci](https://discourse.julialang.org/u/fksayaci)\
**Post date:** [September 22, 2021, 3:56pm UTC](https://discourse.julialang.org/t/can-not-install-pycall-and-pyplot-on-hpc-cluster/68466/16 "2021-09-22T15:56:27Z")

</div>

Yes, I see `python/2.7.15(default)` when I type `module avail`.

Are you saying that it would be enough to write `module load julia python` in the job script (.sh file) to get everything correctly working? Or, would I still have to replicate what I did above (open Julia through terminal, and write `ENV["PYTHON"]=""` and `Pkg.build("PyCall")`?

---

<div class="post-metadata">

**Author:** ![mkitti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkitti/32/12459_2.png) [@mkitti](https://discourse.julialang.org/u/mkitti)\
**Post date:** [September 22, 2021, 9:44pm UTC](https://discourse.julialang.org/t/can-not-install-pycall-and-pyplot-on-hpc-cluster/68466/17 "2021-09-22T21:44:35Z")

</div>

Python 2.7.15? Sure there is a more recent version available.