Bug in LinearAlgebra + Plots-PyPlot combo on intel i9-9900?

I have two computers with intel i9-9900 (new HP dekstop at work + Lenovo home computer), giving the same error on a simple problem – while my laptop (i7) and old work desktop (Dell, Xeon processor?) works fine. Here is the essence of the weird problem:

As you see, the rank is computed incorrectly after a plot, while if I repeat the computation, the answer is correct.

Now, consider the case of using the GR backend:

Some more info:

  1. I use OpenBLAS
  2. The incorrect computation of the rank (and SVD…) doesn’t show up for all matrix sizes – for smaller values of m, the problem may disappear.
  3. On my old (Dell, Xeon?) and new (HP, i9-9900 high power) work desktops, I have installed exactly the same Julia packages and package versions with Julia v 1.4.2. The installation of other software may vary, though. The above code works with PyPlot on my old desktop but not on my new desktop.
  4. The Julia packages on my home computer (Lenovo, i9-9900 low power) and my laptop (Microsoft, i7) are probably different, and I think I use Julia v. 1.4.0 on my laptop + Julia v. 1.4.2 on my (new) home computer. The above code works with PyPlot on my laptop (i7), but not with my new (i9) home computer.
  5. The same error shows up if I run the code in VScode.

I understand that some other Julia user had a similar problem on an i9-9900 gaming computer running model fit in the Polynomials package (which is where I, too, came across the problem). To me, it seems like the problem relates to some interaction between LinearAlgebra and PyPlot on some processors…??

Is anyone with an i9-9900 (or other i9 processors) able to recreate this bug?

do you get the same problem if you use PyPlots directly instead of using it as a back-end for Plots ?

what happens if you take your example and put it in a function ?

1 Like

Like this?

Seems like this resolves the problem?? (I haven’t used PyPlot directly before).

Does this indicate that the problem is in the Plots ↔ pyplot backend interface?

It is not 100% clear to me what I should put in a function. You mean to put rang in a function, or both plot and rang?

It sure won’t help to just put rand in a function – the whole problem showed up because of an error message in the fit function of Polynomials.jl (fit uses pinv, and hence crashes because of a problem with svd).

I mean put the entirety of your example in a function. Your screen shot makes it looks like your are doing this from the REPL.

I always use PyPlot directly. Since the Plots package is doing intermediate processing and is macro-heavy (I think) and then hands it off to the backend, i was wondering if the plots processing was having a side-effect.

That was the reason for also suggesting to put the whole example in a single function, since that will force all the variables to be local in scope and get everything out of the global environment.

also, it’s usually best to create a self-contained function and then insert it into the discussion in text as a code block using ``` so people can try it out easily instead of having to type it in from a screen shot, e.g.

using PyPlot
using LinearAlgebra

function test1()
  m = 15
  ...
end

etc...

Something like this (run on my Lenovo i9 machine)?

I’m not sure why 2 figures are plotted, or more precisely: I’m not sure why the first figure is plotted. If I skip the display(fig) statement, I get the following:


So – it seems like it is the rendering of the plot that causes the problem.

yes. that’s the idea.

as i said i don’t know plots at all, so i’m not sure why you are seeing 2 plots.

this is a very odd problem, and if it’s reproducible, which it appears to be, it seems like a serious bug of some sort, and probably not a CPU related problem.

OK – function test on my laptop (Surface Book 2, with i7 processor) – with pyplot():

Three observations:

  1. The rank function works on the i7 processor.
  2. Why two plots??
  3. Why does the first plot appear in the middle of the first println() command?

Using the gr() gives the following result on the i7:

OK – here, there is only one plot. And the rank command works. But why does the plot appear before the rank(A):5 printout?

I know it is dangerous to generalize based on testing on 4 computers. But it appears like:

  1. On all machines (? – have only tested on private i7 laptop + private i9 PC), it looks like the pyplot backend prints two plots, while only one is expected.
  2. On both i9 processor machines I have tested, the rank() function computes the wrong result for certain matrices when I use the pyplot() backend prior to the rank() command. Non-i9 machines compute the correct value with the rank() function.
  3. Using the gr() backend for Plots, only one plot is printed (as expected, but perhaps in an unexpected sequence). AND, issuing the plot command prior to calling the rank() function does not mess up the rank result.

OK – why don’t I switch to the gr() backend? After all, the gr() backend tends to give somewhat crisper plots, which is good. I’m still holding back to switch to gr() because:

  • The gr() backend has poorer support for LaTeX commands than pyplot(). I really look forward to when the JuliaCon 2020 announced extended support for LaTeX in GR is implemented in the backend.
  • The gr() backend supports fewer plot keywords. Hopefully this will also improve.

In any way, I hope someone can do a more thorough check on the problems with the pyplot() backend in relation to the LinearAlgebra package.

Are you using an MKL-based build for linear algebra? There seems to be a conflict with a linear-algebra library loaded by NumPy:

2 Likes

I’m using OpenBLAS. I have not installed MKL.jl; just using the standard installation of Julia.

I tried to install MKL.jl on my work desktop a couple of days ago in the hope that this might solve the problem. That lead to some “Terminal error” or something, and just stopped the Julia session, so I removed Julia completely with all packages, and reinstalled it.

FWIW I do not reproduce this with a 9900k (OpenBLAS) and Julia 1.5.0:

1 Like

Hm… Thanks for reporting. I’ll check v. 1.5.0 when it is released, and follow up.

Have you tried to set the threads number according to your actual cpu cores?
core i9-9900 has 8 physical cores. So, please check the following command:

LinearAlgebra.BLAS.set_num_threads(8)

1 Like

Setting the number of threads doesn’t change the result. On my i9-9900 home PC:

I reported the issue in https://github.com/JuliaPy/PyPlot.jl/issues/477, and I can reproduce the issue that @BLI reports here.

I am not using MKL.jl so I don’t think library conflict is the cause. I am on a laptop with i5-8265U.

1 Like

Hm. I use OpenBLAS on all computers. OK – it also happens on non-i9 processors.

Looks like you’re running in a Jupyter notebook - does the issue also occur in the REPL? If not, might be a problem with jupyter and not plots/pyplot.

1 Like

I get exactly the same behavior when I run the code in VScode. (See point 5 in my list of “some more info” in my first entry.)

I have now removed every trace of Julia 1.4.2 + Jupyter + conda/Python from my i9-9900 work desktop computer, downloaded and installed Julia 1.5.0 + added packages Plots and PyPlot. No other packages. Still does not work. In Jupyterlab:

In VScode:

using Plots; pyplot()
using LinearAlgebra
#
function test1()
    m = 15
    A = rand(m,5)
    B = rand(m,5)
    println("rank(A):", rank(A))
    plot(sin,size=(100,100)) |> display
    println("rank(B):", rank(B))
    println("rank(B):",rank(B))
end
#
test1()

with REPL response:

julia> using Plots; pyplot()
Plots.PyPlotBackend()

julia> using LinearAlgebra

rank(A):5
rank(B):0
rank(B):5