Using PyCall with a non default Julia managed Conda Environment

New to Julia
, , ,
1

Hi Julia Community,

I’m working on a project where I want to use PyCall, but I’d like to avoid the default Julia-managed Conda environment. Instead, I want to specify a custom environment. Here’s what I’m trying to achieve:

using Pkg, Conda
env = :mymagicenv
Conda.create(env)
Conda.add("matplotlib", env) # Example package
#### What is the correct way to tell PyCall to use the Conda environment `env`?
Pkg.build("PyCall")

I’ve attempted the following approach:

using Pkg, Conda
env = :mymagicenv
Conda.create(env)
Conda.add("matplotlib", env)
ENV["PYTHON"] = ""
ENV["ROOTENV"] = env
using PyCall
mpl = pyimport("matplotlib")

However, this results in the following error:

ERROR: PyError (PyImport_ImportModule
The Python package matplotlib could not be imported by pyimport. Usually this means
that you did not install matplotlib in the Python version being used by PyCall.
PyCall is currently configured to use the Julia-specific Python distribution
installed by the Conda.jl package. To install the matplotlib module, you can
use `pyimport_conda("matplotlib", PKG)`, where PKG is the Anaconda
package that contains the module matplotlib, or alternatively you can use the
Conda package directly (via `using Conda` followed by `Conda.add` etcetera).
Alternatively, if you want to use a different Python distribution on your
system, such as a system-wide Python (as opposed to the Julia-specific Python),
you can re-configure PyCall with that Python. As explained in the PyCall
documentation, set ENV["PYTHON"] to the path/name of the python executable
you want to use, run Pkg.build("PyCall"), and re-launch Julia.
)
ModuleNotFoundError("No module named 'matplotlib'")

Additionally, after encountering this error, attempting to use the up key to recall previous commands in the REPL causes Julia to crash with an access violation error:

Please submit a bug report with steps to reproduce this fault, and any error messages that follow (in their entirety). Thanks.
Exception: EXCEPTION_ACCESS_VIOLATION at 0x7ff9724ca7ac -- PyObject_Free at C:\Users\xxx\.julia\conda\3\x86_64\python312.dll (unknown line)
in expression starting at none:0
PyObject_Free at C:\Users\xxx\.julia\conda\3\x86_64\python312.dll (unknown line)
Allocations: 12468331 (Pool: 12466362; Big: 1969); GC: 24

Please submit a bug report with steps to reproduce this fault, and any error messages that follow (in their entirety). Thanks.
Exception: EXCEPTION_ACCESS_VIOLATION at 0x7ff972601c44 -- Py_FinalizeEx at C:\Users\xxx\.julia\conda\3\x86_64\python312.dll (unknown line)
in expression starting at none:0
Py_FinalizeEx at C:\Users\xxx\.julia\conda\3\x86_64\python312.dll (unknown line)
Py_Finalize at C:\Users\xxx\.julia\packages\PyCall\1gn3u\src\pyinit.jl:125
jfptr_Py_Finalize_7012 at C:\Users\xxx\.julia\compiled\v1.11\PyCall\GkzkC_Yysyl.dll (unknown line)
_atexit at .\initdefs.jl:451
jfptr__atexit_71301.1 at C:\Users\xxx\.julia\juliaup\julia-1.11.6+0.x64.w64.mingw32\lib\julia\sys.dll (unknown line)
jl_apply at C:/workdir/src\julia.h:2157 [inlined]
ijl_atexit_hook at C:/workdir/src\init.c:271
ijl_exit at C:/workdir/src\init.c:207
jl_exception_handler at C:/workdir/src\signals-win.c:337 [inlined]
jl_exception_handler at C:/workdir/src\signals-win.c:229
__julia_personality at C:/workdir/src\win32_ucontext.c:28
_chkstk at C:\WINDOWS\SYSTEM32\ntdll.dll (unknown line)
RtlWow64GetCurrentCpuArea at C:\WINDOWS\SYSTEM32\ntdll.dll (unknown line)
KiUserExceptionDispatcher at C:\WINDOWS\SYSTEM32\ntdll.dll (unknown line)
PyObject_Free at C:\Users\xxx\.julia\conda\3\x86_64\python312.dll (unknown line)
Allocations: 12468331 (Pool: 12466362; Big: 1969); GC: 24

Question:
How can I properly configure PyCall to use a specific Julia-managed Conda environment instead of the default one?

2

According to my understanding, when using PyCall you have two options:

  • using the default Julia Python environment
  • using a custom environment, not managed by Julia

If you want to use different, Julia managed Python environments per Julia projects you probably have to use PythonCall.jl. Unless there is a workaround I am not aware of.

1 Like