How are CPU/core resources utilized in a software model with multiple processes (i.e. data separation), with each process having multiple threads (i.e. data sharing): will the sum of tasks (aggregated over the processes) use the same thread pool and scheduler, or will each process have it’s own thread pool and scheduler (the later of which seems suboptimal to me from a CPU resources point of view)?
Somewhere else I read this:
“”"
Multithreading can be more efficient than the multiprocessing model by taking advantage of shared resources. However, multiprocessing can be simpler due to a clear separation of resources. That said these distinctions are not so simple since there are ways to separate resources between threads and share resources between processes.
“”"
but that still doesn’t answer my question.
In the future, please start your own topic instead of commenting on topic that was solved years ago. The previous participants don’t need the unexpected notification and are less likely to respond after so long.
As for your question, it helps to think about how a few broad layers interact:
CPUs present virtual cores to the operating system (OS) as an abstraction of the physical cores.
In return, the OS schedules OS threads on available virtual cores. The scheduler is typically preemptive, interrupting threads to give other threads some time.
A typical OS manages multiple processes, allotting memory and at least 1 OS thread to each one. In particular, a Julia process can be created with a fixed number of threads.
However, OS-level multithreading poses some problems:
Within one language runtime, people generally want to control when a thread stops and when another starts. If thread A needs thread B to complete some work, then interrupting B just to find out A still isn’t ready is a waste of time.
Interruption requires caching and loading significant CPU states, which adds runtime overhead to many situations that don’t need it.
This isn’t practically relevant to Julia, but old OSs may not even support multithreading. Some computers don’t have OSs at all.
So, some languages bring their own version of threads with their own names; Julia has Tasks. A Julia process has its own scheduler for assigning tasks to its allotted OS threads, which by extension may run in parallel on virtual cores. This scheduler is cooperative, meaning a task yields at deterministic and more efficient points to other tasks.
So bringing it all back to this, each Julia process indeed has its own threads and task scheduler, but there is only one OS thread scheduler so you don’t really have to worry about cores idling. If anything, you’d have to worry about background processes taking time on cores away from Julia, especially on Windows. Different Julia processes cannot pool threads or share a Task scheduler for the simple reason that processes cannot share threads or memory to begin with; at best, processes can coordinate through multiprocessing APIs like Distributed.