# \#gradient

**URL:** https://discourse.julialang.org/tag/gradient/1118.md

[Latest](https://discourse.julialang.org/latest.md) · [Categories](https://discourse.julialang.org/categories.md) · [Tags](https://discourse.julialang.org/tags.md)

---

## [How to ignore minibatches with NaN gradients optimizing a hybrid LUX model using Optimization.jl](https://discourse.julialang.org/t/how-to-ignore-minibatches-with-nan-gradients-optimizing-a-hybrid-lux-model-using-optimization-jl/132615)

<div class="topic-metadata">

**Author:** [@progtw1](https://discourse.julialang.org/u/progtw1)\
**Replies:** 1\
**Last updated:** [October 24, 2025, 1:17pm UTC](https://discourse.julialang.org/t/how-to-ignore-minibatches-with-nan-gradients-optimizing-a-hybrid-lux-model-using-optimization-jl/132615 "2025-10-24T13:17:45Z")

</div>

I try following the LUX tutorial on fitting an ML model using Optimization.jl for fitting a hybrid model, where the LUX model predicts some parameters of a process-based model, the LUX model application is just a call in…

---

## [Just-in-time gradient optimization idea](https://discourse.julialang.org/t/just-in-time-gradient-optimization-idea/131170)

<div class="topic-metadata">

**Author:** [@Tarny\_GG\_Channie](https://discourse.julialang.org/u/Tarny_GG_Channie)\
**Replies:** 2\
**Last updated:** [July 29, 2025, 10:17pm UTC](https://discourse.julialang.org/t/just-in-time-gradient-optimization-idea/131170 "2025-07-29T22:17:28Z")

</div>

Disclaimer: The idea might look simple in theory, but it might be very hard to implement. Imagine you had a machine learning model comprising multiple batches of parameters. Normally, you differentiate through the entir…

---

## [Rrule for zygote](https://discourse.julialang.org/t/rrule-for-zygote/131069)

<div class="topic-metadata">

**Author:** [@mary](https://discourse.julialang.org/u/mary)\
**Replies:** 7\
**Last updated:** [July 29, 2025, 1:37am UTC](https://discourse.julialang.org/t/rrule-for-zygote/131069 "2025-07-29T01:37:23Z")

</div>

Hi, I am using rrule for gradient calculation in zygote. I defined a function which calculates the gradients and I just call that function inside the entire gradient calculation. Can zygote handle this type of rule? fun…

---

## [Stuck differentiating through multidimensional, potentially large and stiff ODE solution](https://discourse.julialang.org/t/stuck-differentiating-through-multidimensional-potentially-large-and-stiff-ode-solution/125651)

<div class="topic-metadata">

**Author:** [@Vitor\_Patricio\_Canta](https://discourse.julialang.org/u/Vitor_Patricio_Canta)\
**Replies:** 0\
**Last updated:** [February 7, 2025, 11:52am UTC](https://discourse.julialang.org/t/stuck-differentiating-through-multidimensional-potentially-large-and-stiff-ode-solution/125651 "2025-02-07T11:52:36Z")

</div>

Hi, I am solving 1D advective dispersive and reactive transport models as Ordinary Differential Equations. To speed things up, I am using Symbolics to detect the sparsity pattern or modelingtoolkitize to reduce allocati…

---

## [Band filled by vertical gradient in Makie.jl](https://discourse.julialang.org/t/band-filled-by-vertical-gradient-in-makie-jl/124987)

<div class="topic-metadata">

**Author:** [@albertomercurio](https://discourse.julialang.org/u/albertomercurio)\
**Replies:** 4\
**Last updated:** [January 20, 2025, 10:43pm UTC](https://discourse.julialang.org/t/band-filled-by-vertical-gradient-in-makie-jl/124987 "2025-01-20T22:43:55Z")

</div>

Hello, I would like to make a band filled by a vertical gradient using CairoMakie.jl. The result should be similar to this example. How can I do it in a clear way using CairoMakie.jl?

---

## [Gradient using ForwardDiff.jl not working](https://discourse.julialang.org/t/gradient-using-forwarddiff-jl-not-working/123193)

<div class="topic-metadata">

**Author:** [@miguelborrero](https://discourse.julialang.org/u/miguelborrero)\
**Replies:** 2\
**Last updated:** [November 28, 2024, 4:57pm UTC](https://discourse.julialang.org/t/gradient-using-forwarddiff-jl-not-working/123193 "2024-11-28T16:57:32Z")

</div>

Hi there, I wrote the following log-likelihood function for which I want to try out ForwardDiff.jl to compute the Jacobian. I have it also by hand but just wanted to try it out and check the differences. However I run i…

---

## [ZeroTangent definition](https://discourse.julialang.org/t/zerotangent-definition/118069)

<div class="topic-metadata">

**Author:** [@mary](https://discourse.julialang.org/u/mary)\
**Replies:** 2\
**Last updated:** [August 12, 2024, 4:55am UTC](https://discourse.julialang.org/t/zerotangent-definition/118069 "2024-08-12T04:55:02Z")

</div>

Hi, I am using chainrulecore for gradient calculation and rrule. I am trying to follow tutorial 18 of gridap. I cant understand the ZeroTangent in the code why we are using that and what it is suppose to do. I’ve read s…

---

## [Zygote.gradient does not work with AbstractGPs.CustomMean](https://discourse.julialang.org/t/zygote-gradient-does-not-work-with-abstractgps-custommean/87815)

<div class="topic-metadata">

**Author:** [@soldasim](https://discourse.julialang.org/u/soldasim)\
**Replies:** 9\
**Last updated:** [August 1, 2024, 6:06am UTC](https://discourse.julialang.org/t/zygote-gradient-does-not-work-with-abstractgps-custommean/87815 "2024-08-01T06:06:37Z")

</div>

Hello, I am trying to use an AbstractGPs.FiniteGP with Turing and I am having problems with Zygote.gradient calculation of the GP hyperparameters. For some reason, Zygote fails to calculate the gradient when using an Ab…

---

## [Custom gradients for function. Resource for rrule and frule](https://discourse.julialang.org/t/custom-gradients-for-function-resource-for-rrule-and-frule/117154)

<div class="topic-metadata">

**Author:** [@KapilKhanal](https://discourse.julialang.org/u/KapilKhanal)\
**Replies:** 18\
**Last updated:** [July 20, 2024, 4:37pm UTC](https://discourse.julialang.org/t/custom-gradients-for-function-resource-for-rrule-and-frule/117154 "2024-07-20T16:37:16Z")

</div>

I am trying to create a custom gradients for a greens function whose term by term direct differentiation is not that accurate and we have a derivation of it’s derivatives and would like to use those. For the simple MWE e…

---

## [NLopt Constraint](https://discourse.julialang.org/t/nlopt-constraint/114856)

<div class="topic-metadata">

**Author:** [@mary](https://discourse.julialang.org/u/mary)\
**Replies:** 4\
**Last updated:** [May 28, 2024, 10:27pm UTC](https://discourse.julialang.org/t/nlopt-constraint/114856 "2024-05-28T22:27:28Z")

</div>

Hi, I am using Nlopt for optimization. my objective function depends on three variables like x1,x2,x3 also I have a constraint which depends on all three variable. I want to do the optimization just for x3 variable. shou…

---

## [Nlopt forced stop](https://discourse.julialang.org/t/nlopt-forced-stop/114727)

<div class="topic-metadata">

**Author:** [@mary](https://discourse.julialang.org/u/mary)\
**Replies:** 2\
**Last updated:** [May 26, 2024, 1:05am UTC](https://discourse.julialang.org/t/nlopt-forced-stop/114727 "2024-05-26T01:05:00Z")

</div>

Hi, I am using Nlopt for a gradient based optimization. It stopes after 4 iterations and returns forced stop and the output is not converged. Could you please tell me how can I find the problem which makes optimizer to f…

---

## [How to construct a matrix of gradients depending on nodal locations in gridap](https://discourse.julialang.org/t/how-to-construct-a-matrix-of-gradients-depending-on-nodal-locations-in-gridap/113255)

<div class="topic-metadata">

**Author:** [@mary](https://discourse.julialang.org/u/mary)\
**Replies:** 0\
**Last updated:** [April 19, 2024, 9:52pm UTC](https://discourse.julialang.org/t/how-to-construct-a-matrix-of-gradients-depending-on-nodal-locations-in-gridap/113255 "2024-04-19T21:52:55Z")

</div>

Hi, I want to calculate the gradients of residuals(displacement) with respect to another vector. based on mathematical calculations, as residuals are a vector this gradient should be a matrix. This matrix should has val…

---

## [What are \`autodiff\_deferred\` and \`autodiff\_thunk\` for in Enzyme?](https://discourse.julialang.org/t/what-are-autodiff-deferred-and-autodiff-thunk-for-in-enzyme/112065)

<div class="topic-metadata">

**Author:** [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Replies:** 23\
**Last updated:** [April 7, 2024, 5:37am UTC](https://discourse.julialang.org/t/what-are-autodiff-deferred-and-autodiff-thunk-for-in-enzyme/112065 "2024-04-07T05:37:00Z")

</div>

I think I understand the basic autodiff function (see this post) but it has three variants: autodiff\_deferred autodiff\_thunk (which seems broken on 1.11) autodiff\_deferred\_thunk Is there someone who can explain to me: …

---

## [Gradients of strain](https://discourse.julialang.org/t/gradients-of-strain/112344)

<div class="topic-metadata">

**Author:** [@mary](https://discourse.julialang.org/u/mary)\
**Replies:** 0\
**Last updated:** [March 31, 2024, 4:09am UTC](https://discourse.julialang.org/t/gradients-of-strain/112344 "2024-03-31T04:09:37Z")

</div>

HI, I want to take the derivative of strain with respect to displacement without using the the shape functions and find the gradients in cells in gridap (I can not figure out this derivative mathematically). I used this…

---

## [Gradient check by zygote](https://discourse.julialang.org/t/gradient-check-by-zygote/109225)

<div class="topic-metadata">

**Author:** [@mary](https://discourse.julialang.org/u/mary)\
**Replies:** 6\
**Last updated:** [January 31, 2024, 10:29pm UTC](https://discourse.julialang.org/t/gradient-check-by-zygote/109225 "2024-01-31T22:29:00Z")

</div>

Hi I am using Julia Gridap library to calculate gradients. I wrote the rules for gradients but when i check the gradients with running finite difference manually. they are different. I check equations multiple time and …

---

## [Taking gradients of a matrix exponential](https://discourse.julialang.org/t/taking-gradients-of-a-matrix-exponential/107865)

<div class="topic-metadata">

**Author:** [@Bizzi](https://discourse.julialang.org/u/Bizzi)\
**Replies:** 10\
**Last updated:** [January 2, 2024, 3:48pm UTC](https://discourse.julialang.org/t/taking-gradients-of-a-matrix-exponential/107865 "2024-01-02T15:48:45Z")

</div>

I’m having trouble differentiating simple matrix exponentials. ForwardDiff and ReverseDiff can’t really do it, while Zygote’s result is type-unstable. using Zygote, ForwardDiff, ReverseDiff M = rand(2,2) ReverseDiff.gr…

---

## [Memory allocations with Zygote](https://discourse.julialang.org/t/memory-allocations-with-zygote/106236)

<div class="topic-metadata">

**Author:** [@Nikos\_Gianniotis](https://discourse.julialang.org/u/Nikos_Gianniotis)\
**Replies:** 8\
**Last updated:** [November 15, 2023, 7:40pm UTC](https://discourse.julialang.org/t/memory-allocations-with-zygote/106236 "2023-11-15T19:40:46Z")

</div>

I have noticed that when I use Zygote to do automatic differentiation in my code (typically I use Zygote to obtain the gradient of my loss function and pass it to Opim.optimize to minimise the loss), I often get high num…

---

## [Gradient accumulation using chainrulecore](https://discourse.julialang.org/t/gradient-accumulation-using-chainrulecore/104208)

<div class="topic-metadata">

**Author:** [@mary](https://discourse.julialang.org/u/mary)\
**Replies:** 1\
**Last updated:** [September 25, 2023, 2:20pm UTC](https://discourse.julialang.org/t/gradient-accumulation-using-chainrulecore/104208 "2023-09-25T14:20:15Z")

</div>

Hi, I am using chainrulecore and zygote and rrule to define my accumulation derivation formulation. the derivative of my function is ∑\_(i=1)^n▒〖w^t da/dr u〗 in order to solve this i have defined below code. Note that…

---

## [Optimization.jl, DataInterpolations.jl and Gradients](https://discourse.julialang.org/t/optimization-jl-datainterpolations-jl-and-gradients/97676)

<div class="topic-metadata">

**Author:** [@astrobc1](https://discourse.julialang.org/u/astrobc1)\
**Replies:** 4\
**Last updated:** [April 21, 2023, 7:27pm UTC](https://discourse.julialang.org/t/optimization-jl-datainterpolations-jl-and-gradients/97676 "2023-04-21T19:27:03Z")

</div>

Hello - This is sort of a follow-up to this post here which didn’t receive attention. Below I provide a more explicit example of what I’m trying to do with interpolating and gradients during optimization. I understand th…

---

## [Avoiding mutation in time course simulations - Zygote](https://discourse.julialang.org/t/avoiding-mutation-in-time-course-simulations-zygote/97141)

<div class="topic-metadata">

**Author:** [@Bill\_Holmes](https://discourse.julialang.org/u/Bill_Holmes)\
**Replies:** 1\
**Last updated:** [April 5, 2023, 11:15pm UTC](https://discourse.julialang.org/t/avoiding-mutation-in-time-course-simulations-zygote/97141 "2023-04-05T23:15:07Z")

</div>

Hello all. Yes, this is another question about avoiding mutation when using Zygote for AD. I am working on a problem where I will eventually need to differentiate through a time dependent simulation. As a toy problem con…

---

## [Error with gradient function in quantum reinforcement learning algorithm](https://discourse.julialang.org/t/error-with-gradient-function-in-quantum-reinforcement-learning-algorithm/96787)

<div class="topic-metadata">

**Author:** [@SatvikDuddukuru](https://discourse.julialang.org/u/SatvikDuddukuru)\
**Replies:** 0\
**Last updated:** [March 29, 2023, 4:26pm UTC](https://discourse.julialang.org/t/error-with-gradient-function-in-quantum-reinforcement-learning-algorithm/96787 "2023-03-29T16:26:22Z")

</div>

Hello. I am trying to implement the REINFORCE algorithm with parametrized quantum circuits from this Python tutorial (Parametrized Quantum Circuits for Reinforcement Learning | TensorFlow Quantum). To do this, I’ve imp…

---

## [Problem with training a spiking neural network with Flux](https://discourse.julialang.org/t/problem-with-training-a-spiking-neural-network-with-flux/95149)

<div class="topic-metadata">

**Author:** [@Daniele\_Linaro](https://discourse.julialang.org/u/Daniele_Linaro)\
**Replies:** 7\
**Last updated:** [March 6, 2023, 4:30pm UTC](https://discourse.julialang.org/t/problem-with-training-a-spiking-neural-network-with-flux/95149 "2023-03-06T16:30:53Z")

</div>

Hello, I am trying to use Flux to train a simple spiking neural network to perform input classification. The network has 100 inputs that represent incoming spike streams: half of the inputs are “active” (i.e., they rece…

---

## [Automatic gradient ∼10x slower to evaluate than the primal computation](https://discourse.julialang.org/t/automatic-gradient-10x-slower-to-evaluate-than-the-primal-computation/93925)

<div class="topic-metadata">

**Author:** [@user22](https://discourse.julialang.org/u/user22)\
**Replies:** 2\
**Last updated:** [February 3, 2023, 6:15am UTC](https://discourse.julialang.org/t/automatic-gradient-10x-slower-to-evaluate-than-the-primal-computation/93925 "2023-02-03T06:15:44Z")

</div>

I need to evaluate a basic dense and a few-layer neural network with up to tens of thousands of inputs at once. The outputs will be then forwarded to another function, which returns a scalar. However, the performance of …

---

## [Gradient of 2D splines on irregular grid](https://discourse.julialang.org/t/gradient-of-2d-splines-on-irregular-grid/87575)

<div class="topic-metadata">

**Author:** [@Thomas\_Mikaelsen](https://discourse.julialang.org/u/Thomas_Mikaelsen)\
**Replies:** 2\
**Last updated:** [September 21, 2022, 2:29pm UTC](https://discourse.julialang.org/t/gradient-of-2d-splines-on-irregular-grid/87575 "2022-09-21T14:29:33Z")

</div>

Hello. I’m looking to (1) interpolate data where the x-axis is a log-spaced grid, the y-axis is a regular spaced grid and there is an array A(x,y) recording the function values in each point; and (2) take the gradient o…

---

## [How to compute hessian of simulated mle](https://discourse.julialang.org/t/how-to-compute-hessian-of-simulated-mle/87998)

<div class="topic-metadata">

**Author:** [@structural](https://discourse.julialang.org/u/structural)\
**Replies:** 7\
**Last updated:** [September 30, 2022, 11:13am UTC](https://discourse.julialang.org/t/how-to-compute-hessian-of-simulated-mle/87998 "2022-09-30T11:13:45Z")

</div>

I need to estimate parameters (a,b) and their corresponding standard errors using simulated MLE. I can successfully estimate the parameters, but I fail to get the SEs. In particular, when I try to obtain the hessian from…

---

## [Gradient vector of a matrix of geographical extent](https://discourse.julialang.org/t/gradient-vector-of-a-matrix-of-geographical-extent/87318)

<div class="topic-metadata">

**Author:** [@ManuelFossa](https://discourse.julialang.org/u/ManuelFossa)\
**Replies:** 3\
**Last updated:** [September 19, 2022, 12:48pm UTC](https://discourse.julialang.org/t/gradient-vector-of-a-matrix-of-geographical-extent/87318 "2022-09-19T12:48:24Z")

</div>

I have a geographical matrix M of dimensions nLat=181 and nLon=221 , with lats = 30:0.25:90 and lons=-30:0.25:25. I want to compute the gradient vector of M w.r.t to lats , lons. I have tried ForwardDiff , Iterations …

---

## [ForwardDiff.jl : ERROR: LoadError: MethodError: convert(...) is ambiguous](https://discourse.julialang.org/t/forwarddiff-jl-error-loaderror-methoderror-convert-is-ambiguous/86132)

<div class="topic-metadata">

**Author:** [@F-YF](https://discourse.julialang.org/u/F-YF)\
**Replies:** 4\
**Last updated:** [August 23, 2022, 7:10am UTC](https://discourse.julialang.org/t/forwarddiff-jl-error-loaderror-methoderror-convert-is-ambiguous/86132 "2022-08-23T07:10:50Z")

</div>

My original intention was to use Zygote to achieve the gradient of gradients, but there were a lot of problems with twice automatic differentiation in reverse mode. So I used Zygote.hessian to achieve the mixing forward…

---

## [Differentiation without explicit function (np.gradient)](https://discourse.julialang.org/t/differentiation-without-explicit-function-np-gradient/57784)

<div class="topic-metadata">

**Author:** [@Salmon](https://discourse.julialang.org/u/Salmon)\
**Replies:** 8\
**Last updated:** [June 12, 2021, 9:14am UTC](https://discourse.julialang.org/t/differentiation-without-explicit-function-np-gradient/57784 "2021-06-12T09:14:46Z")

</div>

Hi everyone, I am currently looking to evaluate some numerical data and need a method similar to numpy.gradient. All the packages, I have found seem to assume that I have some continuous function that I can evaluate at…

---

## [What is the right way to compute Gradient of function of 2 variables with Auto Differentiation?](https://discourse.julialang.org/t/what-is-the-right-way-to-compute-gradient-of-function-of-2-variables-with-auto-differentiation/15138)

<div class="topic-metadata">

**Author:** [@Andrey.Borzunov](https://discourse.julialang.org/u/Andrey.Borzunov)\
**Replies:** 6\
**Last updated:** [September 18, 2018, 8:03pm UTC](https://discourse.julialang.org/t/what-is-the-right-way-to-compute-gradient-of-function-of-2-variables-with-auto-differentiation/15138 "2018-09-18T20:03:42Z")

</div>

I’m terribly confused with number of packages that provide autodiff functionalities and it’s peculiarity. I’m required to compute gradient of multivariable function (e.g. f(x,y), where x,y are Numbers). I found that Au…