# \#ad

**URL:** https://discourse.julialang.org/tag/ad/584.md

[Latest](https://discourse.julialang.org/latest.md) · [Categories](https://discourse.julialang.org/categories.md) · [Tags](https://discourse.julialang.org/tags.md)

---

## [Custom rule for an implicit function in Enzyme](https://discourse.julialang.org/t/custom-rule-for-an-implicit-function-in-enzyme/136259)

<div class="topic-metadata">

**Author:** [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Replies:** 3\
**Last updated:** [March 25, 2026, 12:01pm UTC](https://discourse.julialang.org/t/custom-rule-for-an-implicit-function-in-enzyme/136259 "2026-03-25T12:01:04Z")

</div>

Consider an implicit function y = f(x) defined via g(x, y) where x, y \\in \\mathbb{R}^n. The user supplies f!(y, x) and g!(r, x, y) as callables, which are wrapped in a struct SquareImplicitFunction{F,G} f!::F …

---

## [Automatic differentiation of function using \`LinearProblem\`](https://discourse.julialang.org/t/automatic-differentiation-of-function-using-linearproblem/136063)

<div class="topic-metadata">

**Author:** [@NoFishLikeIan](https://discourse.julialang.org/u/NoFishLikeIan)\
**Replies:** 8\
**Last updated:** [March 22, 2026, 2:08pm UTC](https://discourse.julialang.org/t/automatic-differentiation-of-function-using-linearproblem/136063 "2026-03-22T14:08:42Z")

</div>

I am trying to optimise a function f: \\mathbb{R}^n \\to \\mathbb{R} which internally solves a linear problem, informally something like f(w) = g(x) where x solves A(w) x= b(x). To do this I am using LinearSolve and automa…

---

## [Testing and tolerances for implicitly calculated functions and their AD](https://discourse.julialang.org/t/testing-and-tolerances-for-implicitly-calculated-functions-and-their-ad/133575)

<div class="topic-metadata">

**Author:** [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Replies:** 3\
**Last updated:** [December 2, 2025, 12:49pm UTC](https://discourse.julialang.org/t/testing-and-tolerances-for-implicitly-calculated-functions-and-their-ad/133575 "2025-12-02T12:49:17Z")

</div>

To fix ideas, suppose we have a function x = g(p) defined by f(x(p), p) = 0, both arguments are scalars. The numerical implementation involves finding an x such that | f(x, p)| \\le \\mathrm{tol}, either by Newton’s metho…

---

## [Adjoint sensitivities for non-numeric types in ModelingToolkit.jl](https://discourse.julialang.org/t/adjoint-sensitivities-for-non-numeric-types-in-modelingtoolkit-jl/132082)

<div class="topic-metadata">

**Author:** [@mxp](https://discourse.julialang.org/u/mxp)\
**Replies:** 2\
**Last updated:** [October 17, 2025, 4:51pm UTC](https://discourse.julialang.org/t/adjoint-sensitivities-for-non-numeric-types-in-modelingtoolkit-jl/132082 "2025-10-17T16:51:26Z")

</div>

I’d like to use non-numeric types in ModelingToolkit.jl, but with adjoint sensitivities. When using DifferentialEquations.jl there is support for this through SciMLStructures.jl as described here. So I assumed this would…

---

## [Efficient automatic differentation for Julia version \`jax.scan\`?](https://discourse.julialang.org/t/efficient-automatic-differentation-for-julia-version-jax-scan/132853)

<div class="topic-metadata">

**Author:** [@roflmaostc](https://discourse.julialang.org/u/roflmaostc)\
**Replies:** 4\
**Last updated:** [October 3, 2025, 4:14pm UTC](https://discourse.julialang.org/t/efficient-automatic-differentation-for-julia-version-jax-scan/132853 "2025-10-03T16:14:03Z")

</div>

Hi, I am using in some JAX array code jax.lax.scan which implements something like this in my example: function jax\_lax\_scan(f, x; accumulator\_init) acc = accumulator\_init for i in axes(x, 1) acc = …

---

## [Which direction: DifferentiatonInterface, Enzyme, Zygote with CUDA and FFTs?](https://discourse.julialang.org/t/which-direction-differentiatoninterface-enzyme-zygote-with-cuda-and-ffts/132225)

<div class="topic-metadata">

**Author:** [@roflmaostc](https://discourse.julialang.org/u/roflmaostc)\
**Replies:** 20\
**Last updated:** [September 17, 2025, 6:51am UTC](https://discourse.julialang.org/t/which-direction-differentiatoninterface-enzyme-zygote-with-cuda-and-ffts/132225 "2025-09-17T06:51:26Z")

</div>

Hi! For the third time, I am trying to implement some wave optics algorithms in Julia but each time I start, I suffer a bit from the current automatic differentiation status. In principle, my functions look like: func…

---

## [Automatic differentiation of loglikelihood function. Am I doing it right?](https://discourse.julialang.org/t/automatic-differentiation-of-loglikelihood-function-am-i-doing-it-right/132157)

<div class="topic-metadata">

**Author:** [@giovannitinervia9](https://discourse.julialang.org/u/giovannitinervia9)\
**Replies:** 5\
**Last updated:** [September 8, 2025, 3:56pm UTC](https://discourse.julialang.org/t/automatic-differentiation-of-loglikelihood-function-am-i-doing-it-right/132157 "2025-09-08T15:56:15Z")

</div>

Hi everyone! I’m a statistician and R user taking my first steps with Julia. I have extensive experience with R, and this summer I spent my free time reimplementing the R package gamlss as a personal exercise. Julia has…

---

## [Reverse rule in Enzyme for an implicit function](https://discourse.julialang.org/t/reverse-rule-in-enzyme-for-an-implicit-function/131327)

<div class="topic-metadata">

**Author:** [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Replies:** 2\
**Last updated:** [August 4, 2025, 1:19pm UTC](https://discourse.julialang.org/t/reverse-rule-in-enzyme-for-an-implicit-function/131327 "2025-08-04T13:19:08Z")

</div>

Suppose H is the cdf of a distribution, A, B are real numbers, and p is defined by p = H(p A + (1 - p) B) This can be implemented simply in Julia as using Roots, Distributions struct Problem{T,S} H::T A::S …

---

## [Support for adjoint sensitivities with \`SciMLStructures\` in ODE problems](https://discourse.julialang.org/t/support-for-adjoint-sensitivities-with-scimlstructures-in-ode-problems/131049)

<div class="topic-metadata">

**Author:** [@albangossard](https://discourse.julialang.org/u/albangossard)\
**Replies:** 2\
**Last updated:** [July 25, 2025, 6:15pm UTC](https://discourse.julialang.org/t/support-for-adjoint-sensitivities-with-scimlstructures-in-ode-problems/131049 "2025-07-25T18:15:02Z")

</div>

Hi all, In complex geoscientific models, it’s common to have many simulation parameters that need to be passed into the ODE function. At the same time, we often want to compute gradients with respect to some of these pa…

---

## [Multi-argument Jacobian and gradient](https://discourse.julialang.org/t/multi-argument-jacobian-and-gradient/129029)

<div class="topic-metadata">

**Author:** [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Replies:** 7\
**Last updated:** [May 21, 2025, 12:04pm UTC](https://discourse.julialang.org/t/multi-argument-jacobian-and-gradient/129029 "2025-05-21T12:04:31Z")

</div>

I have a function that takes multiple arguments (all of them vectors), and I want to obtain the Jacobian in each parameter (or, similarly, the gradient if the function maps to \\mathbb{R}). Is it best to unpack/pack like…

---

## [How to think about the cost of vJ and Jv (pullback and pushforward)](https://discourse.julialang.org/t/how-to-think-about-the-cost-of-vj-and-jv-pullback-and-pushforward/129021)

<div class="topic-metadata">

**Author:** [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Replies:** 6\
**Last updated:** [May 15, 2025, 8:58am UTC](https://discourse.julialang.org/t/how-to-think-about-the-cost-of-vj-and-jv-pullback-and-pushforward/129021 "2025-05-15T08:58:49Z")

</div>

In a calculation involving an f: R^n \\to R^m function, where n ranges from 50 to a few hundred and m \\ge n with a similar scale, I have the option of either calculating the Jacobian J once and saving it, or calcula…

---

## [Help to use AD with Optimization.jl](https://discourse.julialang.org/t/help-to-use-ad-with-optimization-jl/128866)

<div class="topic-metadata">

**Author:** [@Maucejo](https://discourse.julialang.org/u/Maucejo)\
**Replies:** 5\
**Last updated:** [May 9, 2025, 2:40pm UTC](https://discourse.julialang.org/t/help-to-use-ad-with-optimization-jl/128866 "2025-05-09T14:40:52Z")

</div>

Hi all, I need your advice to use automatic differentiation with Optimization.jl. The problem - I try to find the optimal variance of a process noise covariance matrix of a standard Kalman filter. Currently, I use grad…

---

## [Debugging non-finite Jacobian elements (w/ ForwardDiff)](https://discourse.julialang.org/t/debugging-non-finite-jacobian-elements-w-forwarddiff/127448)

<div class="topic-metadata">

**Author:** [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Replies:** 4\
**Last updated:** [April 8, 2025, 12:02pm UTC](https://discourse.julialang.org/t/debugging-non-finite-jacobian-elements-w-forwarddiff/127448 "2025-04-08T12:02:04Z")

</div>

I have a rather complex calculation that maps a vector of floats to a longer vector of floats. At some inputs, ForwardDiff gives me a Jacobian where some elements are NaN. Inputs are of course finite, and so is the func…

---

## [Implicit differentiation of rootfinding problem (w/ numerical issues)](https://discourse.julialang.org/t/implicit-differentiation-of-rootfinding-problem-w-numerical-issues/126293)

<div class="topic-metadata">

**Author:** [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Replies:** 13\
**Last updated:** [February 26, 2025, 3:01pm UTC](https://discourse.julialang.org/t/implicit-differentiation-of-rootfinding-problem-w-numerical-issues/126293 "2025-02-26T15:01:40Z")

</div>

I have written an algorithm that solves x = \\log\\left( \\sum\_i \\exp(a\_i + z\_i y) \\right) for y given x, a\_i, z\_i. It uses Newton’s method, which is the trivial part, the difficulty was getting a cheap but reasonable in…

---

## [Differentiating through non-negative least squares solver?](https://discourse.julialang.org/t/differentiating-through-non-negative-least-squares-solver/125641)

<div class="topic-metadata">

**Author:** [@ElOceanografo](https://discourse.julialang.org/u/ElOceanografo)\
**Replies:** 9\
**Last updated:** [February 19, 2025, 10:55am UTC](https://discourse.julialang.org/t/differentiating-through-non-negative-least-squares-solver/125641 "2025-02-19T10:55:18Z")

</div>

Hi, I’m wondering if there’s a better way to get the gradient of a function like this one: using NonNegLeastSquares, DifferentiationInterface import FiniteDiff function f(aa, b) A = reshape(aa, size(b, 1), :) x…

---

## [ExaModels optimizer API for JuMP has convergence issue](https://discourse.julialang.org/t/examodels-optimizer-api-for-jump-has-convergence-issue/126032)

<div class="topic-metadata">

**Author:** [@KSepetanc](https://discourse.julialang.org/u/KSepetanc)\
**Replies:** 5\
**Last updated:** [February 19, 2025, 12:30am UTC](https://discourse.julialang.org/t/examodels-optimizer-api-for-jump-has-convergence-issue/126032 "2025-02-19T00:30:18Z")

</div>

It is my understanding that convergence should not depend on selection of AD package. I have tested many AD options on the same model and only one option has different convergence (number of iterations and reported value…

---

## [Errors when running a Universal Differential Equation (UDE)](https://discourse.julialang.org/t/errors-when-running-a-universal-differential-equation-ude/125273)

<div class="topic-metadata">

**Author:** [@Ashima\_Kalathingal](https://discourse.julialang.org/u/Ashima_Kalathingal)\
**Replies:** 8\
**Last updated:** [February 8, 2025, 12:50pm UTC](https://discourse.julialang.org/t/errors-when-running-a-universal-differential-equation-ude/125273 "2025-02-08T12:50:58Z")

</div>

Hello, I am building a UDE as a part of my work in Julia. I am using the following example as reference https://docs.sciml.ai/Overview/stable/showcase/missing\_physics/ Unfortunately I am getting a warning message and …

---

## [Solver and AD advice for nonlinear LS problem](https://discourse.julialang.org/t/solver-and-ad-advice-for-nonlinear-ls-problem/125014)

<div class="topic-metadata">

**Author:** [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Replies:** 5\
**Last updated:** [January 21, 2025, 5:26pm UTC](https://discourse.julialang.org/t/solver-and-ad-advice-for-nonlinear-ls-problem/125014 "2025-01-21T17:26:51Z")

</div>

(This is a vague open-ended question for which I cannot provide an MWE.) I have a nonlinear least squares problem \\min\_x \\| f(x) \\|\_2^2 where x \\in \\mathbb{R}^N, N \\approx 20...40, and f(x) has about 100–200 elements…

---

## [Help with nested ForwardDiff.jl calculation](https://discourse.julialang.org/t/help-with-nested-forwarddiff-jl-calculation/123077)

<div class="topic-metadata">

**Author:** [@longemen3000](https://discourse.julialang.org/u/longemen3000)\
**Replies:** 1\
**Last updated:** [November 26, 2024, 5:51pm UTC](https://discourse.julialang.org/t/help-with-nested-forwarddiff-jl-calculation/123077 "2024-11-26T17:51:00Z")

</div>

i’m trying to solve a non linear system of equations via newton’s method. the problem is along the lines of: f(model::Nothing,a,x) = sum(x)\*a + log(a/sum(x)) function g(model::Nothing,a,x) = ForwardDiff.gradient(z-\> f(m…

---

## [Correct way of computing adjoints/gradients with dense solution of ODE](https://discourse.julialang.org/t/correct-way-of-computing-adjoints-gradients-with-dense-solution-of-ode/121964)

<div class="topic-metadata">

**Author:** [@facusapienza](https://discourse.julialang.org/u/facusapienza)\
**Replies:** 5\
**Last updated:** [October 31, 2024, 6:17pm UTC](https://discourse.julialang.org/t/correct-way-of-computing-adjoints-gradients-with-dense-solution-of-ode/121964 "2024-10-31T18:17:18Z")

</div>

Hi all, I am trying to use the interpolated solution of my ODE for adjoint sensitivity analysis. Since I have a very small system of ODEs (n=3), I am interested in evaluating the performance of continuous adjoint method…

---

## [\[ANN\] DifferentiationInterface - gradients for everyone](https://discourse.julialang.org/t/ann-differentiationinterface-gradients-for-everyone/113644)

<div class="topic-metadata">

**Author:** [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Replies:** 5\
**Last updated:** [October 8, 2024, 10:19am UTC](https://discourse.julialang.org/t/ann-differentiationinterface-gradients-for-everyone/113644 "2024-10-08T10:19:29Z")

</div>

Julia’s composability has rather interesting consequences for its automatic differentiation ecosystem. Whereas Python programmers first choose a backend (like PyTorch or JAX) and then write code that is specifically tail…

---

## [Nested and different AD methods altogether: How to add AD calculations inside my loss function when using neural differential equations?](https://discourse.julialang.org/t/nested-and-different-ad-methods-altogether-how-to-add-ad-calculations-inside-my-loss-function-when-using-neural-differential-equations/108985)

<div class="topic-metadata">

**Author:** [@facusapienza](https://discourse.julialang.org/u/facusapienza)\
**Replies:** 9\
**Last updated:** [September 28, 2024, 12:16am UTC](https://discourse.julialang.org/t/nested-and-different-ad-methods-altogether-how-to-add-ad-calculations-inside-my-loss-function-when-using-neural-differential-equations/108985 "2024-09-28T00:16:46Z")

</div>

Hi all, I am implementing regularization penalties inside Universal Differential Equations (also applicable to Physics-Informed neural networks) where I need to differentiate a (loss) function that includes in its calcu…

---

## [Best/simplest way to calculate sparse Hessian](https://discourse.julialang.org/t/best-simplest-way-to-calculate-sparse-hessian/58629)

<div class="topic-metadata">

**Author:** [@ElOceanografo](https://discourse.julialang.org/u/ElOceanografo)\
**Replies:** 4\
**Last updated:** [June 10, 2024, 7:01am UTC](https://discourse.julialang.org/t/best-simplest-way-to-calculate-sparse-hessian/58629 "2024-06-10T07:01:34Z")

</div>

I’ve got a log-probability density function of multiple variables, and would like to integrate it using a Laplace approximation. To do this I need to find the function’s maximum and calculate the Hessian at that point. …

---

## [How to: High-performance differentiable programming with broad AD-library support?](https://discourse.julialang.org/t/how-to-high-performance-differentiable-programming-with-broad-ad-library-support/113884)

<div class="topic-metadata">

**Author:** [@ThummeTo](https://discourse.julialang.org/u/ThummeTo)\
**Replies:** 13\
**Last updated:** [May 6, 2024, 3:35pm UTC](https://discourse.julialang.org/t/how-to-high-performance-differentiable-programming-with-broad-ad-library-support/113884 "2024-05-06T15:35:28Z")

</div>

Hi at all, I am currently struggling with three (competing?) goals on code design. I want code, that: (1) is fast, (b) is differentiable and (c) differentiable by current (and future) AD-libraries (like at least Forward…

---

## [Nested AD with Lux etc](https://discourse.julialang.org/t/nested-ad-with-lux-etc/113573)

<div class="topic-metadata">

**Author:** [@avikpal](https://discourse.julialang.org/u/avikpal)\
**Replies:** 26\
**Last updated:** [May 1, 2024, 6:05am UTC](https://discourse.julialang.org/t/nested-ad-with-lux-etc/113573 "2024-05-01T06:05:44Z")

</div>

Nested AD (Starting v0.5.38) Starting v0.5.38, Lux automatically captures some common possibilities of AD calls inside loss functions on lux layers and converts them into a faster version to do a JVP over gradient (inste…

---

## [Allocation-free higher-order derivatives of R^n-\>R](https://discourse.julialang.org/t/allocation-free-higher-order-derivatives-of-r-n-r/112829)

<div class="topic-metadata">

**Author:** [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Replies:** 4\
**Last updated:** [April 12, 2024, 3:26am UTC](https://discourse.julialang.org/t/allocation-free-higher-order-derivatives-of-r-n-r/112829 "2024-04-12T03:26:16Z")

</div>

This question describes a particular AD application, I am either looking for a solution, someone telling me why it is not a good idea, or potential collaborators who are interested in the same problem, which comes up fre…

---

## [ForwardDiff.jl returns NaNs for calculating gradient of simulation model](https://discourse.julialang.org/t/forwarddiff-jl-returns-nans-for-calculating-gradient-of-simulation-model/109796)

<div class="topic-metadata">

**Author:** [@fnoessler](https://discourse.julialang.org/u/fnoessler)\
**Replies:** 2\
**Last updated:** [February 8, 2024, 12:45pm UTC](https://discourse.julialang.org/t/forwarddiff-jl-returns-nans-for-calculating-gradient-of-simulation-model/109796 "2024-02-08T12:45:28Z")

</div>

Hello, this is a follow up post to Differential evolution MCMC in Julia? I want to try HMC/Nuts (via DynamicHMC.jl or AdvancedHMC.jl) because the algorithms may be more efficient than Differential Evolution MCMC. I tr…

---

## [How to do gradient clipping in Julia for large for loops](https://discourse.julialang.org/t/how-to-do-gradient-clipping-in-julia-for-large-for-loops/108402)

<div class="topic-metadata">

**Author:** [@Devetak](https://discourse.julialang.org/u/Devetak)\
**Replies:** 3\
**Last updated:** [January 18, 2024, 10:10am UTC](https://discourse.julialang.org/t/how-to-do-gradient-clipping-in-julia-for-large-for-loops/108402 "2024-01-18T10:10:53Z")

</div>

Hello. I have a function that has many for loops. For example: function f(x) res = 1.0 for i in 1:1000000 res = res \* x^2 end return res / x end of…

---

## [Zygote error when using Nonconvex.jl: Mutating arrays is not supported](https://discourse.julialang.org/t/zygote-error-when-using-nonconvex-jl-mutating-arrays-is-not-supported/107067)

<div class="topic-metadata">

**Author:** [@AP\_profile](https://discourse.julialang.org/u/AP_profile)\
**Replies:** 2\
**Last updated:** [December 3, 2023, 1:59pm UTC](https://discourse.julialang.org/t/zygote-error-when-using-nonconvex-jl-mutating-arrays-is-not-supported/107067 "2023-12-03T13:59:18Z")

</div>

Hi all, I am new to Julia and trying to use SLSQP Nonconvex. The optimization code is as follows: using Nonconvex Nonconvex.@load NLopt start\_vec = zeros(11) gamma\_vec = range(start=-7, stop=7, length=40) rho = 0.58 t…

---

## [Cannot ForwardDiff through linear least squares via QR](https://discourse.julialang.org/t/cannot-forwarddiff-through-linear-least-squares-via-qr/104257)

<div class="topic-metadata">

**Author:** [@Neodym](https://discourse.julialang.org/u/Neodym)\
**Replies:** 6\
**Last updated:** [September 27, 2023, 6:26am UTC](https://discourse.julialang.org/t/cannot-forwarddiff-through-linear-least-squares-via-qr/104257 "2023-09-27T06:26:39Z")

</div>

Hey, I’d like to get derivatives for a linear least squares problem where the parameter appears in the right hand side: using LinearAlgebra using ForwardDiff A = rand(10, 3) b = rand(10) qr\_fact = qr(A) function f(p…

[Next page](https://discourse.julialang.org/tag/ad/584.md?match_all_tags=true&page=1&tags%5B%5D=ad)