I was reading the latest blog post on multithreading and have a few questions on what applications can benefit from threading.
This is how I visualize threads in my head: Suppose a CPU has two threads. You can spawn tasks on both these threads, but only one of them will be executing at any time. The OS may rapidly switch between threads to provide a feel of parallelism, but infact is only doing one job at a one.
For true parallelism, you require separate, independent cores on the CPU. Julia already provides great support for running parallel tasks through its
Distributed api, such as
So to me, threading dosn’t feel like it should generate speedups in “computational” code. It makes sense when dealing with a GUI and a long-running task. That is, pause the long-running task for a miniscule amount of time, update the mouse pointer on screen, and switch back to the long-running task. How does this help with number crunching, because once a thread stops running there is no number crunching going on.
I’d very much appreciate someone explaining this like ELI-(math grad).
Additional question, if someone has time. My research involves agent-based models and running Monte Carlo simulations. We have access to a cluster (18 nodes with 32 cores each, with hyperthreading enabled – so 64 cores?). The workflow involves using
ClusterManagers to connect to the nodes, use
pmap to spawn 32 independent simulations on each of the 32 cores (per node). I spawn 32 processes per node because as far as I think, spawning 64 would actually slow everything down. Is this correct?
The problem I face is that my “independent” simulations also involve code writing results to disk (or STDOUT) and generating logfiles. This is inefficient because the disk is constantly busy writing from each of the 32 processes. I am hoping to use threads here to send information back to the headnode which sits idle while the simulations are happening. The headnode can receive messages in a sequential manner and write it to disk. Can I use threading here at all?