Incorporating SVD factorization into GPUs with CUDA

[jarroyoe]

I tried using the opnorm function, which would be the most efficient scenario. Sadly, Zygote doesn’t have chain rule rules for it. This link suggested using SVD instead.

Implementation of Spectral Normalization for Machine Learning

The problem is that there is currently no rrule defined for svdvals. A workaround would be to define: function snorm(X) return svd(X).S[1] end Which does a full svd (for which there is an rrule defined)

Power iterations may be an alternative, but I’d be concerned in convergence speed.