# \#zygote

**URL:** https://discourse.julialang.org/tag/zygote/326.md?page=1

[Latest](https://discourse.julialang.org/latest.md) · [Categories](https://discourse.julialang.org/categories.md) · [Tags](https://discourse.julialang.org/tags.md)

**Page:** 2

---

## [\[ANN\] DifferentiationInterface - gradients for everyone](https://discourse.julialang.org/t/ann-differentiationinterface-gradients-for-everyone/113644)

<div class="topic-metadata">

**Author:** [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Replies:** 5\
**Last updated:** [October 8, 2024, 10:19am UTC](https://discourse.julialang.org/t/ann-differentiationinterface-gradients-for-everyone/113644 "2024-10-08T10:19:29Z")

</div>

Julia’s composability has rather interesting consequences for its automatic differentiation ecosystem. Whereas Python programmers first choose a backend (like PyTorch or JAX) and then write code that is specifically tail…

---

## [Avoid storing intermediate results from the forward pass by default](https://discourse.julialang.org/t/avoid-storing-intermediate-results-from-the-forward-pass-by-default/119694)

<div class="topic-metadata">

**Author:** [@Jakub\_Mitura](https://discourse.julialang.org/u/Jakub_Mitura)\
**Replies:** 14\
**Last updated:** [September 29, 2024, 4:10pm UTC](https://discourse.julialang.org/t/avoid-storing-intermediate-results-from-the-forward-pass-by-default/119694 "2024-09-29T16:10:01Z")

</div>

Hello, Is it possible in Zygote to change the default behavior for computing derivatives during backpropagation to use gradient checkpointing? I have a memory-constrained problem with a Lux.jl model that uses Zygote fo…

---

## [Emulate Enzyme.Const with Zygote](https://discourse.julialang.org/t/emulate-enzyme-const-with-zygote/119594)

<div class="topic-metadata">

**Author:** [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Replies:** 1\
**Last updated:** [September 19, 2024, 3:05pm UTC](https://discourse.julialang.org/t/emulate-enzyme-const-with-zygote/119594 "2024-09-19T15:05:39Z")

</div>

In the next release, DifferentiationInterface.jl will accept constant (non-differentiated) arguments c in addition to the active (differentiated) argument x, I’m wondering how to implement this mechanism optimally for ea…

---

## [ChainRules project looking for a new primary maintainer](https://discourse.julialang.org/t/chainrules-project-looking-for-a-new-primary-maintainer/115636)

<div class="topic-metadata">

**Author:** [@oxinabox](https://discourse.julialang.org/u/oxinabox)\
**Replies:** 8\
**Last updated:** [September 3, 2024, 4:21pm UTC](https://discourse.julialang.org/t/chainrules-project-looking-for-a-new-primary-maintainer/115636 "2024-09-03T16:21:44Z")

</div>

Hi all, I have been leading the ChainRules project since juliacon 2019. I find myself in circumstances very similar to what Jarret was when he transferred leadership of it to me then – this is a very good thing. As su…

---

## [How to solve an ODEProblem with u0 in the format of vector{vector} and be able to use it for Zygote differentiation](https://discourse.julialang.org/t/how-to-solve-an-odeproblem-with-u0-in-the-format-of-vector-vector-and-be-able-to-use-it-for-zygote-differentiation/118606)

<div class="topic-metadata">

**Author:** [@chooron](https://discourse.julialang.org/u/chooron)\
**Replies:** 1\
**Last updated:** [August 26, 2024, 1:21pm UTC](https://discourse.julialang.org/t/how-to-solve-an-odeproblem-with-u0-in-the-format-of-vector-vector-and-be-able-to-use-it-for-zygote-differentiation/118606 "2024-08-26T13:21:14Z")

</div>

Hello, I am currently facing an ode problem. The format of u0 in this problem is vector{vector}, for example \[\[1,2\], \[1,2,3\], \[1\]\], the particularity of this u0 is that the length of each element is different, and I cann…

---

## [Zygote.gradient does not work with AbstractGPs.CustomMean](https://discourse.julialang.org/t/zygote-gradient-does-not-work-with-abstractgps-custommean/87815)

<div class="topic-metadata">

**Author:** [@soldasim](https://discourse.julialang.org/u/soldasim)\
**Replies:** 9\
**Last updated:** [August 1, 2024, 6:06am UTC](https://discourse.julialang.org/t/zygote-gradient-does-not-work-with-abstractgps-custommean/87815 "2024-08-01T06:06:37Z")

</div>

Hello, I am trying to use an AbstractGPs.FiniteGP with Turing and I am having problems with Zygote.gradient calculation of the GP hyperparameters. For some reason, Zygote fails to calculate the gradient when using an Ab…

---

## [Questions with building a chainrules for mutating array function](https://discourse.julialang.org/t/questions-with-building-a-chainrules-for-mutating-array-function/117237)

<div class="topic-metadata">

**Author:** [@chooron](https://discourse.julialang.org/u/chooron)\
**Replies:** 2\
**Last updated:** [July 22, 2024, 8:30am UTC](https://discourse.julialang.org/t/questions-with-building-a-chainrules-for-mutating-array-function/117237 "2024-07-22T08:30:42Z")

</div>

I want to build two functions for replacing values in a vector and an array, as shown below: function mutate\_vec(vec::AbstractVector{T}, new\_value::T, idx::Int) where {T} vec\[idx\] = new\_value vec end function m…

---

## [Custom gradients for function. Resource for rrule and frule](https://discourse.julialang.org/t/custom-gradients-for-function-resource-for-rrule-and-frule/117154)

<div class="topic-metadata">

**Author:** [@KapilKhanal](https://discourse.julialang.org/u/KapilKhanal)\
**Replies:** 18\
**Last updated:** [July 20, 2024, 4:37pm UTC](https://discourse.julialang.org/t/custom-gradients-for-function-resource-for-rrule-and-frule/117154 "2024-07-20T16:37:16Z")

</div>

I am trying to create a custom gradients for a greens function whose term by term direct differentiation is not that accurate and we have a derivation of it’s derivatives and would like to use those. For the simple MWE e…

---

## [DataInterpolations.jl and Zygote.jl](https://discourse.julialang.org/t/datainterpolations-jl-and-zygote-jl/113208)

<div class="topic-metadata">

**Author:** [@marcobonici](https://discourse.julialang.org/u/marcobonici)\
**Replies:** 18\
**Last updated:** [July 8, 2024, 12:30am UTC](https://discourse.julialang.org/t/datainterpolations-jl-and-zygote-jl/113208 "2024-07-08T00:30:49Z")

</div>

Hi everyone, I am trying to differentiate through an interpolation (as offered by DataInterpolations.jl. Here is a MWE x = Array(LinRange(0,10,101)) a = 1 b = 2 c = 3 p(a,b,c) = a.+b.\*x.+c.\*x.^2 gradient(b -\> sum(p(a,b…

---

## [Flux chain type unstable when broadcasting inside gradient](https://discourse.julialang.org/t/flux-chain-type-unstable-when-broadcasting-inside-gradient/115741)

<div class="topic-metadata">

**Author:** [@filchristou](https://discourse.julialang.org/u/filchristou)\
**Replies:** 2\
**Last updated:** [June 17, 2024, 2:19pm UTC](https://discourse.julialang.org/t/flux-chain-type-unstable-when-broadcasting-inside-gradient/115741 "2024-06-17T14:19:13Z")

</div>

I’ve been fighting for a couple of days to get my Flux/Zygote autodiff code type stable. I am not sure what’s the problem, but coming up with a MWE, it looks like broadcasting Flux.Chain is problematic ? using Flux, Zy…

---

## [Problems with reverse mode automatic diff (e.g. zygote) with NeuralSDEs](https://discourse.julialang.org/t/problems-with-reverse-mode-automatic-diff-e-g-zygote-with-neuralsdes/113501)

<div class="topic-metadata">

**Author:** [@agelgazzar](https://discourse.julialang.org/u/agelgazzar)\
**Replies:** 2\
**Last updated:** [June 15, 2024, 11:17am UTC](https://discourse.julialang.org/t/problems-with-reverse-mode-automatic-diff-e-g-zygote-with-neuralsdes/113501 "2024-06-15T11:17:57Z")

</div>

Hello, I am having difficulty computing gradients of a neural sdes (either custom or in DiffeqFlux) using Zygote or ReverseDiff. I think it has to do with the shape of the noise during the backward pass but I am not su…

---

## [Zygote.hessian of loss function when defining ODEs with modeling toolkit](https://discourse.julialang.org/t/zygote-hessian-of-loss-function-when-defining-odes-with-modeling-toolkit/115302)

<div class="topic-metadata">

**Author:** [@cefitzg](https://discourse.julialang.org/u/cefitzg)\
**Replies:** 1\
**Last updated:** [June 7, 2024, 2:02am UTC](https://discourse.julialang.org/t/zygote-hessian-of-loss-function-when-defining-odes-with-modeling-toolkit/115302 "2024-06-07T02:02:29Z")

</div>

I’m doing parameter estimation on an ODE system, which I define using modelingtoolkit. I need to generate hessian of cost function with respect to parameters to access fit. No issue with parameter estimation. Loss funct…

---

## [Domain Error when using Zygote to differentiate MultiStart minimization](https://discourse.julialang.org/t/domain-error-when-using-zygote-to-differentiate-multistart-minimization/115013)

<div class="topic-metadata">

**Author:** [@paul188](https://discourse.julialang.org/u/paul188)\
**Replies:** 2\
**Last updated:** [June 3, 2024, 12:05pm UTC](https://discourse.julialang.org/t/domain-error-when-using-zygote-to-differentiate-multistart-minimization/115013 "2024-06-03T12:05:57Z")

</div>

I have the following MWE, with most of the code just copied from the MultiStartOptimization.jl documentation: using Optimization, OptimizationMultistartOptimization, OptimizationNLopt using Zygote using ForwardDiff ros…

---

## [Second-order autodiff: which combinations should work?](https://discourse.julialang.org/t/second-order-autodiff-which-combinations-should-work/114892)

<div class="topic-metadata">

**Author:** [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Replies:** 13\
**Last updated:** [May 30, 2024, 10:13am UTC](https://discourse.julialang.org/t/second-order-autodiff-which-combinations-should-work/114892 "2024-05-30T10:13:09Z")

</div>

Hey there! My package DifferentiationInterface.jl provides the necessary infrastructure for second-order automatic differentiation (e.g. Hessians). It allows you to combine arbitrary pairs of AD backends, but of course …

---

## [Zygote.jl with NamedArrays and adjoint](https://discourse.julialang.org/t/zygote-jl-with-namedarrays-and-adjoint/114707)

<div class="topic-metadata">

**Author:** [@PaterPen](https://discourse.julialang.org/u/PaterPen)\
**Replies:** 2\
**Last updated:** [May 25, 2024, 6:52am UTC](https://discourse.julialang.org/t/zygote-jl-with-namedarrays-and-adjoint/114707 "2024-05-25T06:52:54Z")

</div>

Hi, I can’t get Zygote.jl to work with the adjoint of NamedVectors: using Zygote using NamedArrays foo(a, b, c) = a' \* b \* c function bar() a = NamedArray(\[1.0, 2.0\], \["x", "y"\]) b = NamedArray(\[1.0, 2.0\], \["x…

---

## [Discrepancy between complex gradients calculate with Zygote.jl and Python's Jax](https://discourse.julialang.org/t/discrepancy-between-complex-gradients-calculate-with-zygote-jl-and-pythons-jax/114301)

<div class="topic-metadata">

**Author:** [@F-YF](https://discourse.julialang.org/u/F-YF)\
**Replies:** 10\
**Last updated:** [May 16, 2024, 11:24am UTC](https://discourse.julialang.org/t/discrepancy-between-complex-gradients-calculate-with-zygote-jl-and-pythons-jax/114301 "2024-05-16T11:24:31Z")

</div>

When I translated another python project code into julia code, there was a problem. After unremitting efforts, I finally got the following two sample codes Julia codeusing Zygote using FiniteDifferences function test(x…

---

## [Zygote much slower than JAX for automatic differentiation of energy](https://discourse.julialang.org/t/zygote-much-slower-than-jax-for-automatic-differentiation-of-energy/114239)

<div class="topic-metadata">

**Author:** [@albertomercurio](https://discourse.julialang.org/u/albertomercurio)\
**Replies:** 22\
**Last updated:** [May 15, 2024, 6:31am UTC](https://discourse.julialang.org/t/zygote-much-slower-than-jax-for-automatic-differentiation-of-energy/114239 "2024-05-15T06:31:51Z")

</div>

Hello, I’m new in the neural network field, and I’m starting to study Neural Network Quantum States (for example ground state searching using neural networks). Usually, they are implemented in Python using Jax, but then…

---

## [Zygote differentiation very slow](https://discourse.julialang.org/t/zygote-differentiation-very-slow/112751)

<div class="topic-metadata">

**Author:** [@KapilKhanal](https://discourse.julialang.org/u/KapilKhanal)\
**Replies:** 13\
**Last updated:** [May 12, 2024, 11:39pm UTC](https://discourse.julialang.org/t/zygote-differentiation-very-slow/112751 "2024-05-12T23:39:39Z")

</div>

I have an inner function I apply like this (a simple replicable example pasted here) . The input are chosen to be (5,5) but they are usually much larger. I did not do benchmark as this is not even close to acceptable. It…

---

## [How to: High-performance differentiable programming with broad AD-library support?](https://discourse.julialang.org/t/how-to-high-performance-differentiable-programming-with-broad-ad-library-support/113884)

<div class="topic-metadata">

**Author:** [@ThummeTo](https://discourse.julialang.org/u/ThummeTo)\
**Replies:** 13\
**Last updated:** [May 6, 2024, 3:35pm UTC](https://discourse.julialang.org/t/how-to-high-performance-differentiable-programming-with-broad-ad-library-support/113884 "2024-05-06T15:35:28Z")

</div>

Hi at all, I am currently struggling with three (competing?) goals on code design. I want code, that: (1) is fast, (b) is differentiable and (c) differentiable by current (and future) AD-libraries (like at least Forward…

---

## [Why are Neural SDEs trained with TrackerAdjoint()?](https://discourse.julialang.org/t/why-are-neural-sdes-trained-with-trackeradjoint/113514)

<div class="topic-metadata">

**Author:** [@agelgazzar](https://discourse.julialang.org/u/agelgazzar)\
**Replies:** 1\
**Last updated:** [April 29, 2024, 5:39am UTC](https://discourse.julialang.org/t/why-are-neural-sdes-trained-with-trackeradjoint/113514 "2024-04-29T05:39:31Z")

</div>

I thought that Tracker was mostly replaced by Zygote and Enzyme for Reverse-mode Autodiff in the sciml ecosystem. Why is it still being used for computing the sensitivities in DiffeqFlux?

---

## [How to take the gradient of an ODE system with respect to many data points?](https://discourse.julialang.org/t/how-to-take-the-gradient-of-an-ode-system-with-respect-to-many-data-points/113368)

<div class="topic-metadata">

**Author:** [@ctessum](https://discourse.julialang.org/u/ctessum)\
**Replies:** 17\
**Last updated:** [April 25, 2024, 4:58pm UTC](https://discourse.julialang.org/t/how-to-take-the-gradient-of-an-ode-system-with-respect-to-many-data-points/113368 "2024-04-25T16:58:41Z")

</div>

Hello! I have an ode system (dx = f(x, ξ, k)), and I would like to find optimal values of ξ so that: \\mathrm{argmin}\_\\xi \\left( f(x, \\xi, k) - dx \\right)^2 where \\xi \\in R^{n\\xi} are the parameters I want to tune, x…

---

## [How to differentiate using Zygote in the presence of missing values](https://discourse.julialang.org/t/how-to-differentiate-using-zygote-in-the-presence-of-missing-values/112822)

<div class="topic-metadata">

**Author:** [@Nikos\_Gianniotis](https://discourse.julialang.org/u/Nikos_Gianniotis)\
**Replies:** 0\
**Last updated:** [April 11, 2024, 10:05am UTC](https://discourse.julialang.org/t/how-to-differentiate-using-zygote-in-the-presence-of-missing-values/112822 "2024-04-11T10:05:18Z")

</div>

I have encountered a problem in my code. While I cannot easily write a MWE for my particular case, I thought that perhaps dealing with the following issue would help me further. Let us say, that we have some data y and…

---

## [Challenges with Zygote Differentiation in Multivariate Mixture Models as input noise of an implicit machine learning method](https://discourse.julialang.org/t/challenges-with-zygote-differentiation-in-multivariate-mixture-models-as-input-noise-of-an-implicit-machine-learning-method/112560)

<div class="topic-metadata">

**Author:** [@josemanuel22](https://discourse.julialang.org/u/josemanuel22)\
**Replies:** 10\
**Last updated:** [April 5, 2024, 2:23pm UTC](https://discourse.julialang.org/t/challenges-with-zygote-differentiation-in-multivariate-mixture-models-as-input-noise-of-an-implicit-machine-learning-method/112560 "2024-04-05T14:23:25Z")

</div>

The following implicit machine learning method works well when the input noise is, for example, a multidimensional Gaussian function sliced\_invariant\_statistical\_loss\_optimized\_2(nn\_model, loader, hparams) @assert l…

---

## [Questionable Zygote gradients for quantum optimal control problem](https://discourse.julialang.org/t/questionable-zygote-gradients-for-quantum-optimal-control-problem/112060)

<div class="topic-metadata">

**Author:** [@goerz](https://discourse.julialang.org/u/goerz)\
**Replies:** 6\
**Last updated:** [March 26, 2024, 6:06pm UTC](https://discourse.julialang.org/t/questionable-zygote-gradients-for-quantum-optimal-control-problem/112060 "2024-03-26T18:06:32Z")

</div>

I’m testing out a gradient-based optimization of some (quantum) dynamical system using AD over a simulation with OrdinaryDiffEq. The “quantum” here really just means that the dynamical state is complex-valued, not real-v…

---

## [How do you speed up the linear sparse solver in Zygote?](https://discourse.julialang.org/t/how-do-you-speed-up-the-linear-sparse-solver-in-zygote/111801)

<div class="topic-metadata">

**Author:** [@ToPo](https://discourse.julialang.org/u/ToPo)\
**Replies:** 44\
**Last updated:** [March 24, 2024, 7:18am UTC](https://discourse.julialang.org/t/how-do-you-speed-up-the-linear-sparse-solver-in-zygote/111801 "2024-03-24T07:18:47Z")

</div>

I have confirmed that the standard Julia solver (backslash) can be used in the automatic differentiation of Zygote. Below is a sample program. (This program calculates the gradient.) However, while the forward computat…

---

## [Looking for SciMLSensitivity Example with LinearSolve](https://discourse.julialang.org/t/looking-for-scimlsensitivity-example-with-linearsolve/111900)

<div class="topic-metadata">

**Author:** [@Mark\_Lau](https://discourse.julialang.org/u/Mark_Lau)\
**Replies:** 3\
**Last updated:** [March 20, 2024, 10:20pm UTC](https://discourse.julialang.org/t/looking-for-scimlsensitivity-example-with-linearsolve/111900 "2024-03-20T22:20:39Z")

</div>

I’m new to SciMLSensitivity.jl and am trying to learn the ropes. I wanted to try something simple to find the sensitivities of the solution to a LinearProblem with respect to parameters. I couldn’t find an example of how…

---

## [Using Zygote.jl with IterativeSolvers.jl](https://discourse.julialang.org/t/using-zygote-jl-with-iterativesolvers-jl/107634)

<div class="topic-metadata">

**Author:** [@leespen1](https://discourse.julialang.org/u/leespen1)\
**Replies:** 11\
**Last updated:** [February 25, 2024, 9:08pm UTC](https://discourse.julialang.org/t/using-zygote-jl-with-iterativesolvers-jl/107634 "2024-02-25T21:08:40Z")

</div>

I have a code which solves an initial value problem of ODEs parameterized by a control vector, then uses the solution as an argument to calculate an objective function, which I compute the gradient of (by my own methods…

---

## [Differentiating optimization problem solutions in Julia](https://discourse.julialang.org/t/differentiating-optimization-problem-solutions-in-julia/75988)

<div class="topic-metadata">

**Author:** [@mtfishman](https://discourse.julialang.org/u/mtfishman)\
**Replies:** 9\
**Last updated:** [February 25, 2024, 2:10pm UTC](https://discourse.julialang.org/t/differentiating-optimization-problem-solutions-in-julia/75988 "2024-02-25T14:10:32Z")

</div>

Hello, I was recently made aware of the paper “Efficient and Modular Implicit Differentiation”: which is implemented in Jax here: It is about automatically generating differentiation rules for optimization problem…

---

## [Moving a custom loss function to GPU](https://discourse.julialang.org/t/moving-a-custom-loss-function-to-gpu/109637)

<div class="topic-metadata">

**Author:** [@josemanuel22](https://discourse.julialang.org/u/josemanuel22)\
**Replies:** 5\
**Last updated:** [February 15, 2024, 9:19pm UTC](https://discourse.julialang.org/t/moving-a-custom-loss-function-to-gpu/109637 "2024-02-15T21:19:49Z")

</div>

I have developed the following custom loss function, I am trying to run it on the GPU efficiently but I am unable to do so; I get the following error in the line Matrix(ω' \* yₖ\_batch) ERROR: MethodError: no method match…

---

## [Lux.jl LSTM timeseries prediction: AD / optimization does not start](https://discourse.julialang.org/t/lux-jl-lstm-timeseries-prediction-ad-optimization-does-not-start/107553)

<div class="topic-metadata">

**Author:** [@OliverWallscheid](https://discourse.julialang.org/u/OliverWallscheid)\
**Replies:** 2\
**Last updated:** [February 14, 2024, 10:50pm UTC](https://discourse.julialang.org/t/lux-jl-lstm-timeseries-prediction-ad-optimization-does-not-start/107553 "2024-02-14T22:50:50Z")

</div>

Hey there, I am trying to compile a training session of a simple time series prediction example using a LSTM cell within a Lux chain. My model is model = Lux.Chain(Lux.Dense(2 =\> 2), Lux.Recurrence(Lux.LSTMCell(2 =\> 2…

[Previous page](https://discourse.julialang.org/tag/zygote/326.md?match_all_tags=true&tags%5B%5D=zygote)

[Next page](https://discourse.julialang.org/tag/zygote/326.md?match_all_tags=true&page=2&tags%5B%5D=zygote)
