# Sparse Gradient using Zygote & Optim

**URL:** <https://discourse.julialang.org/t/sparse-gradient-using-zygote-optim/49087>\
**Category:** Optimization (Mathematical)\
**Created:** [October 27, 2020, 2:09am UTC](https://discourse.julialang.org/t/sparse-gradient-using-zygote-optim/49087 "2020-10-27T02:09:29Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![sobhan](https://avatars.discourse-cdn.com/v4/letter/s/258eb7/32.png) [@sobhan](https://discourse.julialang.org/u/sobhan)\
**Post date:** [October 27, 2020, 2:09am UTC](https://discourse.julialang.org/t/sparse-gradient-using-zygote-optim/49087/1 "2020-10-27T02:09:29Z")

</div>

I’m trying to minimize a function and i use Zygote to calculate the derivatives to minimize it with BFGS(from Optim), Can i get sparse gradient/hessiens and use them with optim. I really need to switch to sparse since only \frac{4}{n}(where n is the size of the problem) of the parameters can be non-zero. But even in the simplest case it returns a dense gradient.

As for an MWE

```julia
using Zygote
using SparseArrays

f(x)=sum(x.^2)
gradient(f, spzeros(10, 10))[1]

```

```julia
10×10 Array{Float64,2}:
 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0

```

---

<div class="post-metadata">

**Author:** ![jkbest2](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jkbest2/32/7350_2.png) [@jkbest2](https://discourse.julialang.org/u/jkbest2)\
**Post date:** [November 7, 2020, 8:26pm UTC](https://discourse.julialang.org/t/sparse-gradient-using-zygote-optim/49087/2 "2020-11-07T20:26:32Z")

</div>

If you know which four parameters are non-zero, then pass them as arguments to an anonymous function and assign them to the correct places in your gradient vector. If you don’t know, then is your function actually differentiable?

Otherwise you might look at [SparseDiffTools.jl](https://github.com/JuliaDiff/SparseDiffTools.jl).

---

<div class="post-metadata">

**Author:** ![sobhan](https://avatars.discourse-cdn.com/v4/letter/s/258eb7/32.png) [@sobhan](https://discourse.julialang.org/u/sobhan)\
**Post date:** [November 11, 2020, 3:59pm UTC](https://discourse.julialang.org/t/sparse-gradient-using-zygote-optim/49087/3 "2020-11-11T15:59:31Z")

</div>

that was conclusion too, for the sake of completeness

```julia

function vec2sparse(vec, x)
    return SparseMatrixCSC(x.m, x.n, x.colptr, x.rowval, vec)
end
@adjoint vec2sparse(vec, x) = begin
    return vec2sparse(vec, x), c -> (c.nzval, nothing)
end
Zygote.refresh()

```

this is what i used, thanks!