Distributed computing or Multi-threading on Optimization


I am currently working on an algorithm to solve stochastic optimization problems by decomposing them into scenarios (very similar to Progressive Hedging).

I am not very knowledgeable in the area of task parallelization, so I would like to ask: which methodology is more recommended for solving multiple JuMP models in parallel, “Multi-threading” or “Distributed computing”?

Any recommendations or references to supporting material are appreciated!