Hi, I open this topic because I found that sometimes Flux can use 100% of the CPU, but sometimes it only uses half.
For example, here is two demo model, model A have fewer hidden layers than B. Trainning model A, the CPU% is alway 100%, but in model B, it drop down to about 50%,
using Flux demodata = rand(Float32,10000,128) model_A = Chain( Dense(10000 => 64, relu), Dense(64 => 10000,relu)) model_B = Chain( Dense(10000 => 512, relu), Dense(512 => 218,relu), Dense(218 => 64,relu), Dense(64 => 218,relu), Dense(218 => 512,relu), Dense(512 => 10000,relu)) myLoss(Model,X) = begin Flux.Losses.mse(Model(X),X) end # training model_A ps = Flux.params(model_A); for i in 1:100 Flux.train!(myLoss, ps,[(model_A, demodata)], ADAM() ) end # training model_B ps = Flux.params(model_B); for i in 1:100 Flux.train!(myLoss, ps,[(model_B, demodata)], ADAM() ) end
here is the CPU% when the model A is runing
and this one is the model B 's
I am not an expert in this area, but according to my understanding, model B requires more computational power, so, what limits it to using only 50% CPU? and any suggestion how to improve the CPU% in model B training?
Thank you for your patience in reading
I also found this topic related to Flux and CPU, but it is more about multi-threads support.