Update – using latest Flux. I’d like to optimize a function of the singular values of a matrix. The Flux training loop barfs. Here’s my code using Flux, ForwardDiff, LinearAlgebra, GenericLinearAlgebra m = 20 X = randn(m,m) model = Dense(m,m) loss(X) = sum(GenericLinearAlgebra.svdvals(model(X))) …

Here’s how you can get the gradient working: using Flux, Zygote, ForwardDiff using GenericLinearAlgebra: svdvals svdvals2(X) = Zygote.forwarddiff(svdvals, X) m = 20 X = randn(m,m) model = Dense(m,m) loss(X) = sum(svdvals2(X)) ## nuclear norm loss(X) gradient(loss, X) This plugs in forwarddiff …