Hi all, I already set export JULIA_NUM_THREADS=12 and in REPL I checked Threads.nthreads() =12. But when I ran the training, I only saw Julia(Flux) using 8 CPUs, and, they were not in full power. How can I make Flux use all my CPUs and run in full power? Thanks!

Not sure if this is relevant to your specific case, but this happens to me if my batch_size/sample_count aren’t tuned correctly; usually because the CPU is spending all of it’s time moving things around as opposed to actually computing.

Thank you and any guidance on how to tune it correctly?

Try to increase number of threads that openblas (OPENBLAS_NUM_THREADS) uses, since at the moment Flux does not have any paralelization support.

This is actually a point on which I don’t know if there is a ‘correct’ answer. Choosing batch and sample sizes is usually a bit of trial and error for me. I do have a heuristic that I try to stick to and it has served me well for my purposes. I do tend to use a GPU for my work, so take this with a …

Thank you both @Tomas_Pevny @uadjet . But I am afraid that Flux does not support multi-cpu parallelization. I set both JULIA_NUM_THREADS=12 OPENBLAS_NUM_THREADS=12 and I change the batch size. Still Flux does not use 12 CPUs, and, what’s worse, single CPU sometimes.

Correct, Flux does not itself support multi-CPU parallelism (yet). However, because it uses OpenBLAS for its BLAS operations (like the rest of Julia), and because OpenBLAS does do multithreading sometimes, then you sometimes see multiple CPUs getting used. Whether all 12 of them will get used is up …

@jpsamaroo The problem is I want to run my program in the workstation and Flux does not use all of the CPU power. I found that when I ran Flux on a 64-core machine, it used only 8 of them, making the training process very very slow. I also can only use Flux for very small data sets.

Any chance you can either share your code here, or run your code under profiling (Profile.@profile)? That way we can start to investigate where your bottlenecks are and determine if there’s anything that can be done to improve the CPU utilization.

@jpsamaroo Sure. That would be great help. It’s a very simple regression problem: y=f(x_1,x_2,x_3,x_4,x_5,x_6) The code is simple too: using Flux using Flux: throttle using Base.Iterators: repeated using Pkg using DelimitedFiles X_train = readdlm("X_train.txt", ' ', Float64) y_train = readdlm("y…

How can I make Flux use all my CPUs?

General Usage

jpsamaroo March 15, 2019, 7:58pm 11

I didn’t know this before, but apparently Julia’s OpenBLAS is hard-capped to 16 threads (which may show up as 800% CPU on your system): OpenBLAS: Julia slower than R - #17 by Elrod

Topic		Replies	Views
Flux and cpu cores Machine Learning parallel , multithreading , flux	6	1780	September 1, 2020
Flux.jl and the state of multi-processing Machine Learning	2	1666	February 27, 2019
Can Flux handle multiple GPUs? Machine Learning	16	2651	August 5, 2022
Flux parallel execution Machine Learning flux	3	2801	March 29, 2019
Flux multiple cores New to Julia question	0	308	November 9, 2020

How can I make Flux use all my CPUs?

Related topics