Yes - in Julia they do. In Python they don’t. Python (at least as long as the Global Interpreter Lock is still present) is fundamentally single threaded and so Python “threads” work as you describe: You start them and the Python runtime switches between them but at any given point in time only a single thread is actively working.
Julia has a similar concept to that: Tasks. Basically a Task is a chunk of work. You can create multiple Tasks and Julia will switch between them to minimize total execution (e.g. switch to another Task if the current Task needs to wait for I/O).
Threading build on top of Julia’s Tasks. Basically a thread is a something that can work on completing Task and they can work “truly parallel”, i.e. each thread can use a different CPU core [1]. So if you start Julia with more threads, you can have more Tasks running in parallel.
Another way of achieving “true parallelism” is using multiple processes. This is basically a different layer of abstraction. See when you tell your OS to start a program, it puts everything belonging to the program in a process.
So what is the difference between processes and threads? For starter each Julia process have their own separate threads. But what does that mean in practice? Having multiple processes is much more expensive than having multiple threads. That is because the OS allocates resources for process separately. So each Julia process needs to load the external libraries again[2] and needs to compile the code again. Sharing memory is also harder, since the OS usually prohibits a process from accessing a different process’ memory. So accessing common data has a higher overhead.
The main advantage of using different processes is the possibility of running them on physically different hardware, i.e. on multiple computers. In that sense processes are another layer on top of threads.
Summary:
Tasks allow the queuing of multiple independent pieces of work.- Threads work on
Tasks. So multiple threads allow parallel execution on the same machine (by using multiple cores). - Processes allow distribution of work to multiple physical machines.
Note that by default the threads are not pinned to a specific CPU core, you’d need to use ThreadPinning.jl for that. It can have performance advantages. If you want more on this technicalities please ask
↩︎This is the reason that sometimes you need to use multiple processes though. For example BLAS (linear algebra library) uses it’s own threads and that’s a very complicated story ↩︎