Update – using latest Flux. I’d like to optimize a function of the singular values of a matrix. The Flux training loop barfs. Here’s my code using Flux, ForwardDiff, LinearAlgebra, GenericLinearAlgebra m = 20 X = randn(m,m) model = Dense(m,m) loss(X) = sum(GenericLinearAlgebra.svdvals(model(X))) …

Here’s how you can get the gradient working: using Flux, Zygote, ForwardDiff using GenericLinearAlgebra: svdvals svdvals2(X) = Zygote.forwarddiff(svdvals, X) m = 20 X = randn(m,m) model = Dense(m,m) loss(X) = sum(svdvals2(X)) ## nuclear norm loss(X) gradient(loss, X) This plugs in forwarddiff …

Using Flux to optimize a function of the Singular Values

Specific Domains Optimization (Mathematical)

baggepinnen May 18, 2020, 10:57am 2

Welcome to this forum! See this post

to make the code more readable

2 Likes

Topic		Replies	Views
Need help getting started with FluxML Machine Learning flux	5	1010	November 25, 2019
Unable to calculate the gradient of opnorm/svdvals/svd on gpu? General Usage question	9	74	June 13, 2025
Implementation of Spectral Normalization for Machine Learning General Usage	7	805	February 9, 2022
Help with gradient Machine Learning flux	6	1199	January 31, 2020
Flux differentiation error Machine Learning zygote	19	1685	November 19, 2020

Using Flux to optimize a function of the Singular Values

Related topics