When I run the following code using Flux n = 100_000 p = 50 x = rand(Float32, p, n) y = rand(Float32, n) trdata = Flux.Data.DataLoader(x, y, batchsize=100) m = Chain(Dense(p, 100), Dense(100,100), Dense(100,1)) loss(x, y) = Flux.mse(m(x), y) @time Flux.@epochs 10 Flux.train!(loss, Flux.params(m…

It is because openblas is multi-threaded

Ok, thanks. Does this imply that I should avoid Threads.@threads for ... end in order to train several nets simultaneously?

I think that if you use multi-threadding, it would automatically set the number of threads for OpenBlas to one. But in your case, it can still be a win.

[image] Tomas_Pevny: I think that if you use multi-threadding, it would automatically set the number of threads for OpenBlas to one. You have to do this manually . (When comparing the charts, note the different scales on the x-axis!)

Thank you for the comments. I will make a basic comparison in a few days.

I ran a few tests which confirm that BLAS.set_num_threads(1) should be set. On my system the code ran ~10 times faster. Please see below for codes and results # start Julia with JULIA_NUM_THREADS=1 julia using Flux using BenchmarkTools using LinearAlgebra n = 100_000 p = 50 x = rand(Float32, p, n) …

Flux and cpu cores

Specific Domains Machine Learning

Elrod August 26, 2020, 2:35pm 5

You have to do this manually. (When comparing the charts, note the different scales on the x-axis!)

Topic		Replies	Views
How can I make Flux use all my CPUs? General Usage	10	3273	March 15, 2019
Flux multiple cores New to Julia question	0	309	November 9, 2020
Flux.jl and the state of multi-processing Machine Learning	2	1667	February 27, 2019
Flux parallel execution Machine Learning flux	3	2801	March 29, 2019
Why more BLAS threads take more time Performance threads	2	546	September 9, 2022

Flux and cpu cores

Related topics