I’m using GLM to calculate coefficients for logistic regression. I don’t recall this being an issue when I first developed this script, but recently, I’ve noticed that this script is generating a huge load on my server. The CPU usage goes up to 6000% in some cases as monitored through “top”. I’ve narrowed it down to one function as demonstrated in this MWE
df = DataFrame(:col1 => String, :col2 => Bool)
for i = 1:1000
results = glm(@formula(col2 ~ col1), df, Binomial())
I presume that GLM is internally multi threading this calculation, but is there an option to limit the number of threads internally within GLM to prevent saturating the server resources?
Well, it’s using ~ half the available cores… if I had 2 cores and one of them was being used by Julia, I wouldn’t be worried. If i had 10 cores and 5 of them were in use by Julia… I wouldn’t be worried…
If I have 64 cores and 32 are being used by Julia… should I be worried? I mean, maybe you want to limit it further… sure… but it’s not really a “huge” load for such a server.
In any case, if it’s BLAS that’s the issue, you can use BLAS.set_num_threads() to limit its thread count.
ERROR: UndefVarError: BLAS not defined
I tried to add the BLAS package, but there is no BLAS package to add.
(@v1.8) pkg> add BLAS
Updating registry at `/prj/yeprd/server/julia/PKG/1.6/registries/General.toml`
ERROR: The following package names could not be resolved:
* BLAS (not found in project, manifest or registry)