Hi, I have a project where I need to call python code via PyCall.jl. This python code itself does interact with a third-party library (In this case I use pyspice the control the simulation of multiple circuits). To increase the speed of my script I would like to run multiple threads that each contr…

I don’t have a working PyCall to hand, but with PythonCall the following works fine, and I’d expect PyCall to work just the same: julia> @everywhere using PythonCall julia> @time pmap(1:4) do i __main__, time = pyimport("__main__", "time") time.sleep(4 - i) __main_…

[image] cjdoris: pmap(1:4) do i Note those are different processes (distributed memory), which is totally distinct from different shared-memory “threads” — it’s really crucial to understand the difference if you are doing parallel programming!

The OP specifically mentioned pmap and Distributed so I answered to that (multi-processing). If you want to use multi-threading then the short answer is no, you can’t use PyCall (or PythonCall) with that. The long answer is yes, maybe, if you’re very very careful with the GIL.

Hi, could you please tell me what should be done with the GIL for multithreading?

[image] martinmestre: could you please tell me what should be done with the GIL for multithreading? See Dropping GIL when calling Julia from Python (JuliaCall) · Issue #343 · JuliaPy/PythonCall.jl · GitHub But general if you want to parallelize Python calls it is best to do it from multiple …

It seems that Python can’t be made running in different threads, but there’s a workaround to pin python routines to a fixed thread just in case your application (e.g. a multi-user web application like GenieFramwork), will use multi-threading to handle different user requests. I posted a solution in…

Run multiple python instances with pycall in different threads

General Usage

stevengj Split this topic August 16, 2023, 9:07pm 7

2 posts were split to a new topic: Distributed-memory parallelism on a shared-memory machine

Topic		Replies	Views
Using PyCall on different threads crashes my Julia REPL General Usage question	3	724	November 10, 2021
Using pycall from threads New to Julia pycall , multithreading	1	1455	December 27, 2019
Run Parallel Python in Julia General Usage question	3	961	February 5, 2020
Julia crashed when using PyCall in multiple threads General Usage pycall	14	889	January 24, 2026
Can JuliaCall easily handle multi-threading? General Usage juliacall , pythoncall	14	1174	February 22, 2024

Run multiple python instances with pycall in different threads

Related topics