Flux parallel execution

Hi,

so, this might be a very dumb question but how do I train model in flux.jl in parallel on my cpu?

2 Likes

Same question. MNIST demo in model zoo runs on a single core only. I am even surprised that Flux doesn’t run on multiple cores by default. Isn’t it mainly matrix multiplication which is paralleled by Julia?

Edit:

It does supported by julia. set JULIA_NUM_THREADS env variable will do the trick
I use following before running julia

export JULIA_NUM_THREADS=6
1 Like

I have implemented a manual model-parallel computations, where each model is evaluated on a single thread and on the end their gradient is averaged. It is faster in some cases. But I believe that the overall goal should be to have underlying routines multi-threaded.

The implementation is here.

In Windows you have to set the enviroment variable using the Control Panel.