Build PyCall error

New to Julia
1

Hello everyone, it’s been a while since I had any errors while installing Julia. When Julia tries to install PyCall, it runs the script

.julia/packages/PyCall/tqyST/deps/find_libpython.py

is generates many error messages, of which I think the most important is

File “/home/marllos/.julia/packages/PyCall/BD546/deps/find_libpython.py”, line 33, in
import ctypes.util

When I time give the command “import ctypes.util” directly in pytyon 3.9.4, I get the message:

ModuleNotFoundError: No module named ‘_ctypes’

I appreciate any help.

julia> versioninfo()
Julia Version 1.6.0
Commit f9720dc2eb (2021-03-24 12:55 UTC)
Platform Info:
OS: Linux (x86_64-pc-linux-gnu)
CPU: AMD EPYC 7452 32-Core Processor
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-11.0.1 (ORCJIT, znver2)

shell> lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 20.04.2 LTS
Release Date: 20.04
Codename: focal

2

Can anyone tell me something?

3

If you type import ctypes.util directly in Python and get an error, then your Python installation is broken.

Fix/reinstall your python installation (or let PyCall install its own by setting ENV["PYTHON"]="" before building PyCall).

1 Like
4

The Python installed is the one that came with the default Ubuntu installation. What I could do was apt-get remove python3 and then apt-get install python3, but the error still persisted.

5

Maybe you have some environment variable, like PYTHONPATH, set to a bogus value that is confusing Python.

1 Like
6

I checked, it doesn’t. The OS was installed 3 days ago. The server had not yet been used when I installed Julia.The ENV tip worked, but I still want the default solution, using the Python that comes with Ubuntu, just like the 20 times I installed Julia without errors.

7

Then complain to the OS vendor or your local sysadmin. If import ctypes.util fails when you type it manually in Python, then it is a problem with your Python setup — it has nothing to do with Julia or PyCall.

1 Like
8

See this perhaps:
https://stackoverflow.com/questions/27022373/python3-importerror-no-module-named-ctypes-when-using-value-from-module-mul/41310760#41310760

1 Like
9

I also think that the problem has nothing to do with Julia or PyCall.

My procedure has always been:

  1. ubuntu installation
  2. sudo apt-get install python3-matplotlib libqt5widgets5
  3. installation of Julia
  4. sudo ln -s /opt/julia-1.6.0/bin/julia /usr/local/bin/julia

I did this on Ubuntu 20.04.2 and Ubuntu 21.04 and everything went fine. When I did it again on the latest Ubuntu 20.04.2 it gave a lot of errors:

(@v1.6) pkg> add PyCall
Installing known registries into ~/.julia
Added registry General to ~/.julia/registries/General
Resolving package versions…
Installed MacroTools ───── v0.5.6
Installed VersionParsing ─ v1.2.0
Installed Conda ────────── v1.5.1
Installed JSON ─────────── v0.21.1
Installed Parsers ──────── v1.1.0
Installed PyCall ───────── v1.92.3
Updating ~/.julia/environments/v1.6/Project.toml
[438e738f] + PyCall v1.92.3
Updating ~/.julia/environments/v1.6/Manifest.toml
[8f4d0f93] + Conda v1.5.1
[682c06a0] + JSON v0.21.1
[1914dd2f] + MacroTools v0.5.6
[69de0a69] + Parsers v1.1.0
[438e738f] + PyCall v1.92.3
[81def892] + VersionParsing v1.2.0
[2a0f44e3] + Base64
[ade2ca70] + Dates
[8f399da3] + Libdl
[37e2e46d] + LinearAlgebra
[d6f4376e] + Markdown
[a63ad114] + Mmap
[de0858da] + Printf
[9a3f8284] + Random
[9e88b42a] + Serialization
[4ec0a83e] + Unicode
Building Conda ─→ ~/.julia/scratchspaces/44cfe95a-1eb2-52ea-b672-e2afdf69b78f/6231e40619c15148bcb80aa19d731e629877d762/build.log
Building PyCall → ~/.julia/scratchspaces/44cfe95a-1eb2-52ea-b672-e2afdf69b78f/169bb8ea6b1b143c5cf57df6d34d022a7b60c6db/build.log
ERROR: Error building PyCall:
Traceback (most recent call last):
File “/home/marllos/.julia/packages/PyCall/BD546/deps/find_libpython.py”, line 33, in
import ctypes.util
File “/usr/local/lib/python3.9/ctypes/init.py”, line 8, in
from _ctypes import Union, Structure, Array
ModuleNotFoundError: No module named 'ctypes’
ERROR: LoadError: failed process: Process(setenv(python3 /home/marllos/.julia/packages/PyCall/BD546/deps/find_libpython.py --list-all,[“PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin”, “PWD=/home/marllos”, “XDG_SESSION_CLASS=user”, “LESSOPEN=| /usr/bin/lesspipe %s”, “SHELL=/bin/bash”, “SSH_CONNECTION=200.136.244.11 58299 200.136.245.156 22”, “OPENBLAS_NUM_THREADS=8”, “DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1003/bus”, “MOTD_SHOWN=pam”, “LANG=en_US.UTF-8” … "=/usr/local/bin/julia", “JULIA_LOAD_PATH=@:/tmp/jl_RdnWJ4”, “USER=marllos”, “SSH_TTY=/dev/pts/0”, “LESSCLOSE=/usr/bin/lesspipe %s %s”, “HOME=/home/marllos”, “TERM=xterm”, "LS_COLORS=rs=, …“PYTHONIOENCODING=UTF-8”]), ProcessExited(1)) [1]

Stacktrace:
[1] pipeline_error
@ ./process.jl:525 [inlined]
[2] (::Base.var"#644#645"{Base.Process})()
@ Base ./process.jl:319
[3] iterate(itr::Base.EachLine{Base.PipeEndpoint}, state::Nothing) (repeats 2 times)
@ Base ./io.jl:1014
[4] _collect(cont::UnitRange{Int64}, itr::Base.EachLine{Base.PipeEndpoint}, #unused#::Base.HasEltype, isz::Base.SizeUnknown)
@ Base ./array.jl:613
[5] collect
@ ./array.jl:602 [inlined]
[6] #readlines#340
@ ./io.jl:551 [inlined]
[7] readlines
@ ./io.jl:551 [inlined]
[8] exec_find_libpython
@ ~/.julia/packages/PyCall/BD546/deps/buildutils.jl:28 [inlined]
[9] find_libpython(python::String; _dlopen::typeof(Base.Libc.Libdl.dlopen))
@ Main ~/.julia/packages/PyCall/BD546/deps/buildutils.jl:43
[10] find_libpython(python::String)
@ Main ~/.julia/packages/PyCall/BD546/deps/buildutils.jl:41
[11] top-level scope
@ ~/.julia/packages/PyCall/BD546/deps/build.jl:87
[12] include(fname::String)
@ Base.MainInclude ./client.jl:444
[13] top-level scope
@ none:5
in expression starting at /home/marllos/.julia/packages/PyCall/BD546/deps/build.jl:43

10

I did and it said I already had the latest version.

$ sudo apt-get install libffi-dev
Reading package lists… Done
Building dependency tree
Reading state information… Done
libffi-dev is already the newest version (3.3-4).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

11

Did you clone cpython and do

./configure
make # alternatively `make -j 4` will utilize 4 threads
sudo make altinstall

?

12

Alternatively, did you do what SGJ said:

julia> ENV["PYTHON"]=""

pkg> build PyCall

I’m not sure what’s wrong with Ubuntu’s Python, but apparently you are not the only one with the issue. Doing the above will let PyCall grab a new Python via Conda. Then you don’t have to worry about the state of Ubuntu’s Python or have to build your own.

13

Will each user get a copy of Python via Conda in ~/.julia, or if I do it for myself, does it apply to everyone?

14

One thing I only just noticed is that python3 points to python3.9 (3.9.4). And the “import ctypes.util” command works in python3.8. Just to clarify, I’m not asking for help to fix Julia bugs, but to fix the infrastructure needed for Julia. Thank you.

15

OK, does this work?

julia> ENV["PYTHON"] = "/usr/bin/python3.8" # Use which to locate python3.8

julia> Pkg.build("PyCall")
3 Likes
16

Thank you very much mkitti. Your tip worked very well. I saved it in ~/.julia/config.

17

The link seems to say something about my topic, stating that there is no error in python3.9 regarding ctypes.util.

1 Like
18

I would probably check with the package maintainer for however you obtained Python 3.9.4. For example:

https://launchpad.net/ubuntu/+source/python3.9/+bugs