# KrylovKit svd solver not running properly

**URL:** <https://discourse.julialang.org/t/krylovkit-svd-solver-not-running-properly/28812>\
**Category:** New to Julia\
**Created:** [September 16, 2019, 2:51pm UTC](https://discourse.julialang.org/t/krylovkit-svd-solver-not-running-properly/28812 "2019-09-16T14:51:28Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![iamsuddhasattwa](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/iamsuddhasattwa/32/7441_2.png) [@iamsuddhasattwa](https://discourse.julialang.org/u/iamsuddhasattwa)\
**Post date:** [September 16, 2019, 2:51pm UTC](https://discourse.julialang.org/t/krylovkit-svd-solver-not-running-properly/28812/1 "2019-09-16T14:51:28Z")

</div>

I am encountering a problem with the svd solver of the KylovKit.jl package. I am trying to compute the SVD of a **90000X90000** sparse matrix, with a fraction of **2.4E-4** elements non-zero. However, the following execution

```julia
using KrylovKit;
K=deserialize(open( "K.dat" ));
Lambda,Phi,Gamma,info = KrylovKit.svdsolve(K, 1, krylovdim=3, tol=1E-3, maxiter=100);

```

does not terminate (it has been more than 48 hours). I am looking only for the top SVD, and my tolerance is also very high. The matrix K is close to the identity and its top singular value is 1.

Could suggest any reason why this relatively simple svd application is not working. Is there any known bug ?

Another curious thing is that my computer has 4 processors, but only one of them is being used during the operation.

The sparse matrix K is created from a complicated process involving the computation of a Gaussian kernel over a data-set. So i have not provided those details.

---

<div class="post-metadata">

**Author:** ![juthohaegeman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/juthohaegeman/32/8620_2.png) [@juthohaegeman](https://discourse.julialang.org/u/juthohaegeman)\
**Post date:** [September 16, 2019, 7:29pm UTC](https://discourse.julialang.org/t/krylovkit-svd-solver-not-running-properly/28812/2 "2019-09-16T19:29:20Z")

</div>

The svd solver is probably the least tested. That being said, I notice you have krylovdim=3. Is there any reason for choosing it so so small? You can get more info about the convergence by using the `verbosity` keyword

---

<div class="post-metadata">

**Author:** ![iamsuddhasattwa](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/iamsuddhasattwa/32/7441_2.png) [@iamsuddhasattwa](https://discourse.julialang.org/u/iamsuddhasattwa)\
**Post date:** [September 16, 2019, 8:14pm UTC](https://discourse.julialang.org/t/krylovkit-svd-solver-not-running-properly/28812/3 "2019-09-16T20:14:42Z")

</div>

I was under the impression that it is sufficient for the krylov dimension needs to be 2L+1, where L is the number of singular values/vectors required.  
I will try out with the verbosity command.  
Can you suggest any other tried and tested package. I had earlier reported a problem / bug with the `svds()` routine [here](https://discourse.julialang.org/t/arpackexception-unspecified-arpack-error-when-doing-svd-of-sparse-matrix/22753/3), to which you had suggested `KrylovKit.jl`

---

<div class="post-metadata">

**Author:** ![juthohaegeman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/juthohaegeman/32/8620_2.png) [@juthohaegeman](https://discourse.julialang.org/u/juthohaegeman)\
**Post date:** [September 16, 2019, 8:30pm UTC](https://discourse.julialang.org/t/krylovkit-svd-solver-not-running-properly/28812/4 "2019-09-16T20:30:22Z")

</div>

2L+1 is theoretically sufficient, but I would suggest larger to get faster convergence. `krylovdim` around 20 to 30 seems to be an often made choice.

---

<div class="post-metadata">

**Author:** ![iamsuddhasattwa](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/iamsuddhasattwa/32/7441_2.png) [@iamsuddhasattwa](https://discourse.julialang.org/u/iamsuddhasattwa)\
**Post date:** [September 17, 2019, 3:21pm UTC](https://discourse.julialang.org/t/krylovkit-svd-solver-not-running-properly/28812/5 "2019-09-17T15:21:39Z")

</div>

I now ran the command without specifying the krylov dimension, as follows

`@time Lambda,Phi,Gamma,info = KrylovKit.svdsolve( K, L, verbosity=3, tol=1E-3, maxiter=100 );`

The process has been stuck at the 30th iteration for several hours. Only 1 out 4 processors is still being utilized.

```julia
[ Info: GKL iteration step 1: normres = NaN
[ Info: GKL iteration step 2: normres = NaN
...
[ Info: GKL iteration step 30: normres = NaN

```

Do you know what this could mean ? Also, can you suggest any other svd solver ? svd solving is very crucial for many numeric applications.

---

<div class="post-metadata">

**Author:** ![platawiec](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/platawiec/32/31914_2.png) [@platawiec](https://discourse.julialang.org/u/platawiec)\
**Post date:** [September 17, 2019, 4:50pm UTC](https://discourse.julialang.org/t/krylovkit-svd-solver-not-running-properly/28812/6 "2019-09-17T16:50:29Z")

</div>

You can try `svdl` in `IterativeSolvers`.

[https://juliamath.github.io/IterativeSolvers.jl/dev/svd/svdl/](https://juliamath.github.io/IterativeSolvers.jl/dev/svd/svdl/)

The fact that `normres` is `NaN` is concerning. What does it return when you set `maxiter=1`?

---

<div class="post-metadata">

**Author:** ![iamsuddhasattwa](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/iamsuddhasattwa/32/7441_2.png) [@iamsuddhasattwa](https://discourse.julialang.org/u/iamsuddhasattwa)\
**Post date:** [September 17, 2019, 6:26pm UTC](https://discourse.julialang.org/t/krylovkit-svd-solver-not-running-properly/28812/7 "2019-09-17T18:26:51Z")

</div>

Setting `maxiter=1` still has the same problem :

```julia
julia> @time Lambda,Phi,Gamma,info = KrylovKit.svdsolve( K, 1, verbosity=3, tol=1E-3, maxiter=1 );
[ Info: GKL iteration step 1: normres = NaN
...
[ Info: GKL iteration step 30: normres = NaN

```

The process stops indefinitely after iteration step 30.

---

<div class="post-metadata">

**Author:** ![iamsuddhasattwa](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/iamsuddhasattwa/32/7441_2.png) [@iamsuddhasattwa](https://discourse.julialang.org/u/iamsuddhasattwa)\
**Post date:** [September 17, 2019, 6:37pm UTC](https://discourse.julialang.org/t/krylovkit-svd-solver-not-running-properly/28812/8 "2019-09-17T18:37:29Z")

</div>

Thank you for the suggestion. I looked up the documentation [here](https://juliamath.github.io/IterativeSolvers.jl/dev/svd/svdl/#SVDL-1). I am having travel in deciphering the output. I tried the routine on a simple identity amtrix of size 4.

```julia
A = Diagonal(ones(4));
X,L = IterativeSolvers.svdl(A,nsv=3,vecs="both");

```

The problem is that I do not know how the SVD results are stored in X,L. Upon checking, I find

```julia
julia> X
SVD{Float64,Float64,Array{Float64,2}}(Array{Float64}(4,0), [7.76331, 5.56498, 3.40801], Array{Float64}(0,4))

julia> L
IterativeSolvers.PartialFactorization{Float64,Float64}([0.148432 -0.00588581 … 0.993988 -0.993989; 0.016312 -0.000646821 … 0.109257 -0.109253; 0.000746992 -2.96202e-5 … 0.00503284 -0.0050266; 0.000748214 -2.9669e-5 … 0.00502097 -0.00501883], [1.15233 -0.0327544 … -0.99399 0.99399; 0.126635 -0.00359955 … -0.109242 0.109243; 0.00579912 -0.000164836 … -0.00501194 0.00501381; 0.00580861 -0.000165107 … -0.00501379 0.00501443], [7.76331 0.0 … 0.0 0.0; 0.0 5.56498 … 0.0 0.0; … ; 0.0 0.0 … 2.34803 2.34803; 0.0 0.0 … 0.0 3.34803], 3.348034540367908)

```

I do not see 3 singular values equal to one. Where can I recover the left and right singular vectors from ? The documentation does not indicate that.

In other programs such as Matlab, the documentation clearly provides this basic information. Usually people are interested in both the singular vectors and values.

---

<div class="post-metadata">

**Author:** ![jkbest2](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jkbest2/32/7350_2.png) [@jkbest2](https://discourse.julialang.org/u/jkbest2)\
**Post date:** [September 17, 2019, 6:54pm UTC](https://discourse.julialang.org/t/krylovkit-svd-solver-not-running-properly/28812/9 "2019-09-17T18:54:27Z")

</div>

Your `X` here is an `SVD` object, which is well documented [here](https://docs.julialang.org/en/v1.2/stdlib/LinearAlgebra/#LinearAlgebra.svd).

---

<div class="post-metadata">

**Author:** ![juthohaegeman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/juthohaegeman/32/8620_2.png) [@juthohaegeman](https://discourse.julialang.org/u/juthohaegeman)\
**Post date:** [September 17, 2019, 7:54pm UTC](https://discourse.julialang.org/t/krylovkit-svd-solver-not-running-properly/28812/10 "2019-09-17T19:54:54Z")

</div>

Could you share K.dat ? It should indeed not happen that you get `NaN` for `normres` (and I should probably check for that and error out).

---

<div class="post-metadata">

**Author:** ![iamsuddhasattwa](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/iamsuddhasattwa/32/7441_2.png) [@iamsuddhasattwa](https://discourse.julialang.org/u/iamsuddhasattwa)\
**Post date:** [September 17, 2019, 9:01pm UTC](https://discourse.julialang.org/t/krylovkit-svd-solver-not-running-properly/28812/11 "2019-09-17T21:01:16Z")

</div>

It is a 3GB file, so I shared it on Google drive with you  
[https://drive.google.com/open?id=1d\_v-qyLhaZcfrZ6X0wW3tKgf\_2GRtjvr](https://drive.google.com/open?id=1d_v-qyLhaZcfrZ6X0wW3tKgf_2GRtjvr)

---

<div class="post-metadata">

**Author:** ![iamsuddhasattwa](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/iamsuddhasattwa/32/7441_2.png) [@iamsuddhasattwa](https://discourse.julialang.org/u/iamsuddhasattwa)\
**Post date:** [September 17, 2019, 9:17pm UTC](https://discourse.julialang.org/t/krylovkit-svd-solver-not-running-properly/28812/12 "2019-09-17T21:17:44Z")

</div>

Thank you. I am having some problems with this package, which I have posted on a [separate](https://discourse.julialang.org/t/problems-with-iterativesolvers-svdl-routine/28866) thread

---

<div class="post-metadata">

**Author:** ![juthohaegeman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/juthohaegeman/32/8620_2.png) [@juthohaegeman](https://discourse.julialang.org/u/juthohaegeman)\
**Post date:** [September 17, 2019, 10:15pm UTC](https://discourse.julialang.org/t/krylovkit-svd-solver-not-running-properly/28812/13 "2019-09-17T22:15:44Z")

</div>

Your matrix contains `Inf` values, so any vector you multiply it with will also come to have infinite norm.

---

<div class="post-metadata">

**Author:** ![iamsuddhasattwa](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/iamsuddhasattwa/32/7441_2.png) [@iamsuddhasattwa](https://discourse.julialang.org/u/iamsuddhasattwa)\
**Post date:** [September 18, 2019, 4:17am UTC](https://discourse.julialang.org/t/krylovkit-svd-solver-not-running-properly/28812/14 "2019-09-18T04:17:43Z")

</div>

Thank you for the painstaking effort, that was indeed the problem. It had never occurred to me.