# Zygote much slower than JAX for automatic differentiation of energy

**URL:** <https://discourse.julialang.org/t/zygote-much-slower-than-jax-for-automatic-differentiation-of-energy/114239>\
**Category:** Machine Learning\
**Tags:** performance, flux, zygote, jax, lux\
**Created:** [May 14, 2024, 11:48am UTC](https://discourse.julialang.org/t/zygote-much-slower-than-jax-for-automatic-differentiation-of-energy/114239 "2024-05-14T11:48:32Z")\
**Posts on this page:** 3\
**Page:** 2

<div class="post-metadata">

**Author:** ![avikpal](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/avikpal/32/6550_2.png) [@avikpal](https://discourse.julialang.org/u/avikpal)\
**Post date:** [May 14, 2024, 10:25pm UTC](https://discourse.julialang.org/t/zygote-much-slower-than-jax-for-automatic-differentiation-of-energy/114239/22 "2024-05-14T22:25:05Z")

</div>

What about the code @gdalle posted? That should be quite fast and would work on GPUs with Zygote

---

<div class="post-metadata">

**Author:** ![albertomercurio](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/albertomercurio/32/27051_2.png) [@albertomercurio](https://discourse.julialang.org/u/albertomercurio)\
**Post date:** [May 14, 2024, 11:12pm UTC](https://discourse.julialang.org/t/zygote-much-slower-than-jax-for-automatic-differentiation-of-energy/114239/23 "2024-05-14T23:12:04Z")

</div>

But we understood that Zygote is very slow in this case. It works on GPU, but it is slow.

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/224680_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [May 15, 2024, 6:31am UTC](https://discourse.julialang.org/t/zygote-much-slower-than-jax-for-automatic-differentiation-of-energy/114239/24 "2024-05-15T06:31:51Z")

</div>

Then you might have to wait a few days until a solution on the Zygote side materializes. Turn on notifications for this issue:

> <https://github.com/JuliaDiff/ChainRules.jl/issues/788>
>
> Currently this line https://github.com/JuliaDiff/ChainRules.jl/blob/9f1817a22404…259113e230bef149a54d379a660b/src/rulesets/LinearAlgebra/dense.jl#L38 leads to a loss of structure because it constructs a dense matrix from 2 vectors. We should make this a lazy outer product (like in https://github.com/SciML/LinearSolve.jl/pull/484) such that the \`m x n\` matrix's co-tangent only requires \`O(m + n)\` memory. See https://discourse.julialang.org/t/zygote-jl-how-to-get-the-gradient-of-sparse-matrix/59067/12 for the potential gain and suggested implementation. If the idea gets a green light, I will open a PR.

[Previous page](https://discourse.julialang.org/t/zygote-much-slower-than-jax-for-automatic-differentiation-of-energy/114239.md?page=1)
