# \#forwarddiff

**URL:** https://discourse.julialang.org/tag/forwarddiff/363.md?page=3

[Latest](https://discourse.julialang.org/latest.md) · [Categories](https://discourse.julialang.org/categories.md) · [Tags](https://discourse.julialang.org/tags.md)

**Page:** 4

---

## [Bug in Forward diff for complex valued function](https://discourse.julialang.org/t/bug-in-forward-diff-for-complex-valued-function/100119)

<div class="topic-metadata">

**Author:** [@Veenty](https://discourse.julialang.org/u/Veenty)\
**Replies:** 2\
**Last updated:** [June 9, 2023, 5:02pm UTC](https://discourse.julialang.org/t/bug-in-forward-diff-for-complex-valued-function/100119 "2023-06-09T17:02:58Z")

</div>

I have an extremely simple case scenario r(s) = (1 + 0.3\*cos(8π\*s + 2\*sin(2π\*s)))\*exp(1im\*(2π\*s - π/2)) r′(s) = ForwardDiff.derivative(r, s) r′′(s) = ForwardDiff.derivative(r′, s) h\_ref = 1/(5\*2^9) T\_ref = collect(0:h\_r…

---

## [DiffEqFlux and Threaded Ensemble Problems with ForwardDiff](https://discourse.julialang.org/t/diffeqflux-and-threaded-ensemble-problems-with-forwarddiff/99514)

<div class="topic-metadata">

**Author:** [@Allan\_Baker](https://discourse.julialang.org/u/Allan_Baker)\
**Replies:** 3\
**Last updated:** [May 28, 2023, 6:48pm UTC](https://discourse.julialang.org/t/diffeqflux-and-threaded-ensemble-problems-with-forwarddiff/99514 "2023-05-28T18:48:55Z")

</div>

Hi Julia Speed Guru’s, I’m trying to demonstrate Julia’s cool capabilities for embedding shallow neural networks to select key parameters for an ensemble of differential equations. I can use the ensemble problem fine a…

---

## [Custom rule for differentiating through Newton solver using ForwardDiff: works for gradient, fails for hessian](https://discourse.julialang.org/t/custom-rule-for-differentiating-through-newton-solver-using-forwarddiff-works-for-gradient-fails-for-hessian/93002)

<div class="topic-metadata">

**Author:** [@touste](https://discourse.julialang.org/u/touste)\
**Replies:** 23\
**Last updated:** [May 26, 2023, 5:31pm UTC](https://discourse.julialang.org/t/custom-rule-for-differentiating-through-newton-solver-using-forwarddiff-works-for-gradient-fails-for-hessian/93002 "2023-05-26T17:31:05Z")

</div>

Hi all, I have a function obj(p) that solves for the parameter t of a Bezier curve B(t) so that the arclength is equal to some arbitrary value (2 in this kind-of-minimal example). It uses the Newton’s method. The Bezie…

---

## [ReverseDiff over ForwardDiff behaves strangely with Lux networks (Issue)](https://discourse.julialang.org/t/reversediff-over-forwarddiff-behaves-strangely-with-lux-networks-issue/99175)

<div class="topic-metadata">

**Author:** [@Bizzi](https://discourse.julialang.org/u/Bizzi)\
**Replies:** 3\
**Last updated:** [May 23, 2023, 2:06am UTC](https://discourse.julialang.org/t/reversediff-over-forwarddiff-behaves-strangely-with-lux-networks-issue/99175 "2023-05-23T02:06:13Z")

</div>

Hello everyone. For my work with PINNs, I am trying to use ReverseDiff to calculate the gradient of a loss function that itself contains a ForwardDiff gradient. It is easy enough to do so with explicitly-parametrized dec…

---

## [How to efficiently differentiate backslash operator for sparse matrix?](https://discourse.julialang.org/t/how-to-efficiently-differentiate-backslash-operator-for-sparse-matrix/43778)

<div class="topic-metadata">

**Author:** [@Liang\_Chen](https://discourse.julialang.org/u/Liang_Chen)\
**Replies:** 23\
**Last updated:** [May 1, 2023, 4:21pm UTC](https://discourse.julialang.org/t/how-to-efficiently-differentiate-backslash-operator-for-sparse-matrix/43778 "2023-05-01T16:21:24Z")

</div>

My problem is to calculate the gradient, dL(u)/dp, where L() is a loss function which takes a vector u as the input and outputs a scalar value. The vector p are parameters that defines a large sparse matrix K, such that K…

---

## [How to order axis with categorical variable in AlgebraOfGraphics?](https://discourse.julialang.org/t/how-to-order-axis-with-categorical-variable-in-algebraofgraphics/98027)

<div class="topic-metadata">

**Author:** [@SebastianCallh](https://discourse.julialang.org/u/SebastianCallh)\
**Replies:** 1\
**Last updated:** [April 28, 2023, 11:32am UTC](https://discourse.julialang.org/t/how-to-order-axis-with-categorical-variable-in-algebraofgraphics/98027 "2023-04-28T11:32:16Z")

</div>

Hello! I’m experimenting with AlgebraOfGraphics and stumbled upon something I haven’t been able to do. I looks like the default behaviour when plotting categorical variable on the y axis and continuous on the x axis is t…

---

## [ForwardDiff through StatsBase's fit(Histogram, ...)](https://discourse.julialang.org/t/forwarddiff-through-statsbases-fit-histogram/97892)

<div class="topic-metadata">

**Author:** [@tom-plaa](https://discourse.julialang.org/u/tom-plaa)\
**Replies:** 0\
**Last updated:** [April 25, 2023, 11:25am UTC](https://discourse.julialang.org/t/forwarddiff-through-statsbases-fit-histogram/97892 "2023-04-25T11:25:23Z")

</div>

I have code that is trying to autodifferentiate through the histogram fitting of StatsBase using ForwardDiff, any tips on how to get this to work? I’m hitting TwicePrecision and it’s trying to convert to Float64. Here’s…

---

## [\[Help\] Nested ForwardDiff.jacobian of an Integrals.IntegralProblem](https://discourse.julialang.org/t/help-nested-forwarddiff-jacobian-of-an-integrals-integralproblem/97430)

<div class="topic-metadata">

**Author:** [@aafsar](https://discourse.julialang.org/u/aafsar)\
**Replies:** 2\
**Last updated:** [April 13, 2023, 5:36pm UTC](https://discourse.julialang.org/t/help-nested-forwarddiff-jacobian-of-an-integrals-integralproblem/97430 "2023-04-13T17:36:03Z")

</div>

I have an integration problem (actually vector-valued, but problem here applies to scalar-valued ones as well) as a part of my model which it might be helpful to calculate its Hessian. I follow the documentation of Integ…

---

## [Zygote vs. Forward Diff with Optim](https://discourse.julialang.org/t/zygote-vs-forward-diff-with-optim/96565)

<div class="topic-metadata">

**Author:** [@gnicolosi](https://discourse.julialang.org/u/gnicolosi)\
**Replies:** 4\
**Last updated:** [March 24, 2023, 11:39pm UTC](https://discourse.julialang.org/t/zygote-vs-forward-diff-with-optim/96565 "2023-03-24T23:39:13Z")

</div>

Hi all, I am solving an optimization problem using an Augmented Lagrangian (AL) of a constrained parameterized problem. I have about 400 parameters and the AL spits out a scalar which is to be minimized. I have already …

---

## [Efficiently in-place filling an array of ForwardDiff.Dual](https://discourse.julialang.org/t/efficiently-in-place-filling-an-array-of-forwarddiff-dual/96537)

<div class="topic-metadata">

**Author:** [@cadaverous](https://discourse.julialang.org/u/cadaverous)\
**Replies:** 2\
**Last updated:** [March 24, 2023, 12:35am UTC](https://discourse.julialang.org/t/efficiently-in-place-filling-an-array-of-forwarddiff-dual/96537 "2023-03-24T00:35:29Z")

</div>

I was able to write function to handle arrays of ForwardDiff.Dual, since in theory I should be able to compute the derivative of a certain function much more efficiently than just propagating Dual numbers through it. I h…

---

## [How to optimize Jacobian ForwardDiff.jl](https://discourse.julialang.org/t/how-to-optimize-jacobian-forwarddiff-jl/96177)

<div class="topic-metadata">

**Author:** [@Sergey\_Novak](https://discourse.julialang.org/u/Sergey_Novak)\
**Replies:** 2\
**Last updated:** [March 20, 2023, 8:20am UTC](https://discourse.julialang.org/t/how-to-optimize-jacobian-forwarddiff-jl/96177 "2023-03-20T08:20:41Z")

</div>

Hello, I manual calculate Jacobian for the system function TM(u, p, t) U(y) = p\[8\] + p\[9\] / ( 1.0 + exp( -50.0 \* (y - p\[7\]) ) ) σ(x) = 1.0 / ( 1.0 + exp( -20.0 \* (x-p\[6\]) ) ) du1 = (-u\[1\] + p\[1\] \* log( 1.0 +…

---

## [CUDA atomic add for ForwardDiff duals?](https://discourse.julialang.org/t/cuda-atomic-add-for-forwarddiff-duals/85968)

<div class="topic-metadata">

**Author:** [@Mark\_Lau](https://discourse.julialang.org/u/Mark_Lau)\
**Replies:** 2\
**Last updated:** [August 19, 2022, 5:59pm UTC](https://discourse.julialang.org/t/cuda-atomic-add-for-forwarddiff-duals/85968 "2022-08-19T17:59:41Z")

</div>

Hi everyone, I’ve been trying to auto differentiate some code that runs on the GPU using ForwardDiff. Some of the kernels perform an atomic addition operation via CUDA.@atomic, but that doesn’t seem to be supported for F…

---

## [Best method for (scalar) second derivative with ForwardDiff](https://discourse.julialang.org/t/best-method-for-scalar-second-derivative-with-forwarddiff/95039)

<div class="topic-metadata">

**Author:** [@fph](https://discourse.julialang.org/u/fph)\
**Replies:** 4\
**Last updated:** [February 23, 2023, 5:04pm UTC](https://discourse.julialang.org/t/best-method-for-scalar-second-derivative-with-forwarddiff/95039 "2023-02-23T17:04:23Z")

</div>

What is the best/most efficient method to find the first two derivatives of a scalar function with ForwardDiff? This old post suggests ForwardDiff.derivative(x -\> ForwardDiff.derivative(f, x), 1.0) to compute the seco…

---

## [A hacky guide to using automatic differentiation in nested optimization problems](https://discourse.julialang.org/t/a-hacky-guide-to-using-automatic-differentiation-in-nested-optimization-problems/39123)

<div class="topic-metadata">

**Author:** [@jeffreyesun](https://discourse.julialang.org/u/jeffreyesun)\
**Replies:** 25\
**Last updated:** [February 17, 2023, 3:32pm UTC](https://discourse.julialang.org/t/a-hacky-guide-to-using-automatic-differentiation-in-nested-optimization-problems/39123 "2023-02-17T15:32:11Z")

</div>

Unconstrained Optimization Automatic differentiation is amazing. But models in my field, economics, often include nested optimization problems. A simple example might look like this, where x is a vector or scalar. y = …

---

## [Error with ForwardDiff in Turing](https://discourse.julialang.org/t/error-with-forwarddiff-in-turing/88633)

<div class="topic-metadata">

**Author:** [@astro-kevin](https://discourse.julialang.org/u/astro-kevin)\
**Replies:** 30\
**Last updated:** [February 15, 2023, 12:19pm UTC](https://discourse.julialang.org/t/error-with-forwarddiff-in-turing/88633 "2023-02-15T12:19:34Z")

</div>

When running the sample method from Turing, I run into this error MethodError: no method matching Float64(::ForwardDiff.Dual{ForwardDiff.Tag{Turing.TuringTag, Float64}, Float64, 9}) Closest candidates are: (::Type{T}…

---

## [Models with persistent memory usage](https://discourse.julialang.org/t/models-with-persistent-memory-usage/93215)

<div class="topic-metadata">

**Author:** [@pfarndt](https://discourse.julialang.org/u/pfarndt)\
**Replies:** 3\
**Last updated:** [February 6, 2023, 5:12pm UTC](https://discourse.julialang.org/t/models-with-persistent-memory-usage/93215 "2023-02-06T17:12:16Z")

</div>

I am using Turing a lot but now I have a model which would keep some persistent memory to be computational efficient. A minimal example would be: using Turing, Optim, ForwardDiff mutable struct Acomputer{T \<: Number} …

---

## [Function structure for ForwardDiff jacobian](https://discourse.julialang.org/t/function-structure-for-forwarddiff-jacobian/93991)

<div class="topic-metadata">

**Author:** [@rakshith95](https://discourse.julialang.org/u/rakshith95)\
**Replies:** 0\
**Last updated:** [February 3, 2023, 2:22pm UTC](https://discourse.julialang.org/t/function-structure-for-forwarddiff-jacobian/93991 "2023-02-03T14:22:00Z")

</div>

Hello, I’m working with images, and I’m seeing some strange behaviour on Jacobian computation, so I suspect my function formulation is incorrect. I have two patches in an image, and my function F which returns a vector…

---

## [Automatic differentiation of function with special points](https://discourse.julialang.org/t/automatic-differentiation-of-function-with-special-points/93780)

<div class="topic-metadata">

**Author:** [@rpgowers](https://discourse.julialang.org/u/rpgowers)\
**Replies:** 4\
**Last updated:** [February 1, 2023, 9:50am UTC](https://discourse.julialang.org/t/automatic-differentiation-of-function-with-special-points/93780 "2023-02-01T09:50:06Z")

</div>

I am using ForwardDiff.jl to compute derivatives of functions which often contain functions of the form: f(x,y) = x/\[exp(x/y)-1\] which one can show using L’Hôpital’s rule has the value f(0,y) = y. One way to construc…

---

## [How to tell ForwardDiff what i want it to do?](https://discourse.julialang.org/t/how-to-tell-forwarddiff-what-i-want-it-to-do/93167)

<div class="topic-metadata">

**Author:** [@lrnv](https://discourse.julialang.org/u/lrnv)\
**Replies:** 8\
**Last updated:** [January 19, 2023, 4:13pm UTC](https://discourse.julialang.org/t/how-to-tell-forwarddiff-what-i-want-it-to-do/93167 "2023-01-19T16:13:48Z")

</div>

Hey, This is a follow up to this previous post where I was looking for the right derivatives of the gamma\_inc function. I found them. Now i am trying to tell ForwardDiff to use them… with issues. The function has two a…

---

## [ForwardDiff AD Error using distributed within Turing @model](https://discourse.julialang.org/t/forwarddiff-ad-error-using-distributed-within-turing-model/93111)

<div class="topic-metadata">

**Author:** [@abhishek-venket](https://discourse.julialang.org/u/abhishek-venket)\
**Replies:** 0\
**Last updated:** [January 17, 2023, 7:17pm UTC](https://discourse.julialang.org/t/forwarddiff-ad-error-using-distributed-within-turing-model/93111 "2023-01-17T19:17:00Z")

</div>

I am trying to estimate the posterior distributions of a couple of parameters of a system of ODEs. The probabilistic model is data ~ MvNormal(y\_sim, σ² \* I), where data contains sensor measurements of a quantity of inte…

---

## [ForwardDiff.derivative pointwise](https://discourse.julialang.org/t/forwarddiff-derivative-pointwise/92867)

<div class="topic-metadata">

**Author:** [@c\_sell](https://discourse.julialang.org/u/c_sell)\
**Replies:** 2\
**Last updated:** [January 12, 2023, 1:15pm UTC](https://discourse.julialang.org/t/forwarddiff-derivative-pointwise/92867 "2023-01-12T13:15:48Z")

</div>

Hello, is it possible to use ForwardDiff in the following way. I have a function (a forward diff through a neural network). I want to take the second derivative of the output with respect to the input. If I use n input…

---

## [Are computations with Dual numbers twice as slow as non Duals?](https://discourse.julialang.org/t/are-computations-with-dual-numbers-twice-as-slow-as-non-duals/92502)

<div class="topic-metadata">

**Author:** [@nrontsis](https://discourse.julialang.org/u/nrontsis)\
**Replies:** 2\
**Last updated:** [January 4, 2023, 2:39pm UTC](https://discourse.julialang.org/t/are-computations-with-dual-numbers-twice-as-slow-as-non-duals/92502 "2023-01-04T14:39:08Z")

</div>

After reading this I got the impression that computations with dual numbers are typically expected to be twice as slow as non-duals. The reasoning was that both the normal and dual computations have to calculated at each…

---

## [Declaring ForwardDiff tag directly with a DifferentialEquations integrator / nested function](https://discourse.julialang.org/t/declaring-forwarddiff-tag-directly-with-a-differentialequations-integrator-nested-function/83766)

<div class="topic-metadata">

**Author:** [@DanielVandH](https://discourse.julialang.org/u/DanielVandH)\
**Replies:** 3\
**Last updated:** [December 24, 2022, 11:30pm UTC](https://discourse.julialang.org/t/declaring-forwarddiff-tag-directly-with-a-differentialequations-integrator-nested-function/83766 "2022-12-24T23:30:13Z")

</div>

This is a continuation of an original question of mine (Updating integrator parameter/initial condition with ForwardDiff.Dual types) to be more focused on the actual problem of defining an array/scalar for use with autom…

---

## [ForwardDiff and FunctionWrappers](https://discourse.julialang.org/t/forwarddiff-and-functionwrappers/91130)

<div class="topic-metadata">

**Author:** [@DanielVandH](https://discourse.julialang.org/u/DanielVandH)\
**Replies:** 8\
**Last updated:** [December 3, 2022, 11:21am UTC](https://discourse.julialang.org/t/forwarddiff-and-functionwrappers/91130 "2022-12-03T11:21:14Z")

</div>

I have code that makes use of FunctionWrappers in some places. I need to compute gradients for code that involves these wrappers - is this possible? For example, using FunctionWrappers using ForwardDiff f1 = FunctionWra…

---

## [Using ForwardDiff results in a stack overflow](https://discourse.julialang.org/t/using-forwarddiff-results-in-a-stack-overflow/90399)

<div class="topic-metadata">

**Author:** [@c\_sell](https://discourse.julialang.org/u/c_sell)\
**Replies:** 4\
**Last updated:** [November 18, 2022, 12:16pm UTC](https://discourse.julialang.org/t/using-forwarddiff-results-in-a-stack-overflow/90399 "2022-11-18T12:16:46Z")

</div>

Hi, im using the ForwardDiff package. The following code is only a simplification of the real problem. But I am getting there as well a stack overflow in the line of the LU decomposition. using ForwardDiff, LinearAlgeb…

---

## [Gradient of 2D splines on irregular grid](https://discourse.julialang.org/t/gradient-of-2d-splines-on-irregular-grid/87575)

<div class="topic-metadata">

**Author:** [@Thomas\_Mikaelsen](https://discourse.julialang.org/u/Thomas_Mikaelsen)\
**Replies:** 2\
**Last updated:** [September 21, 2022, 2:29pm UTC](https://discourse.julialang.org/t/gradient-of-2d-splines-on-irregular-grid/87575 "2022-09-21T14:29:33Z")

</div>

Hello. I’m looking to (1) interpolate data where the x-axis is a log-spaced grid, the y-axis is a regular spaced grid and there is an array A(x,y) recording the function values in each point; and (2) take the gradient o…

---

## [T Distribution - Automatic Differentiation on quantile and cdf function](https://discourse.julialang.org/t/t-distribution-automatic-differentiation-on-quantile-and-cdf-function/89434)

<div class="topic-metadata">

**Author:** [@mrVeng](https://discourse.julialang.org/u/mrVeng)\
**Replies:** 9\
**Last updated:** [November 7, 2022, 3:42pm UTC](https://discourse.julialang.org/t/t-distribution-automatic-differentiation-on-quantile-and-cdf-function/89434 "2022-11-07T15:42:02Z")

</div>

Hi there, I need to use the cdf (and quantile) function of the T distribution inside a function that I want to use Automatic Differentiation on. This works fine with almost any univariate distribution from the Distribut…

---

## [Autodiff-ing a function defined by the result of Roots.find\_zero fails](https://discourse.julialang.org/t/autodiff-ing-a-function-defined-by-the-result-of-roots-find-zero-fails/87753)

<div class="topic-metadata">

**Author:** [@joaovgranja](https://discourse.julialang.org/u/joaovgranja)\
**Replies:** 12\
**Last updated:** [September 29, 2022, 9:54am UTC](https://discourse.julialang.org/t/autodiff-ing-a-function-defined-by-the-result-of-roots-find-zero-fails/87753 "2022-09-29T09:54:52Z")

</div>

Hi all! I am trying to ForwardDiff a function that is defined by solving a non-linear equation that depends on parameters. The code below reproduces the error. It first redefines exp by solving log(x) - a = 0 using Roo…

---

## [Nth (univariate) derivative using AD](https://discourse.julialang.org/t/nth-univariate-derivative-using-ad/87504)

<div class="topic-metadata">

**Author:** [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Replies:** 5\
**Last updated:** [September 20, 2022, 9:24pm UTC](https://discourse.julialang.org/t/nth-univariate-derivative-using-ad/87504 "2022-09-20T21:24:58Z")

</div>

I need an n th derivative of a univariate function for testing purposes. I came up with import ForwardDiff function ddn(f, x, ::Val{N}) where {N} fn = foldl((f, \_) -\> x -\> ForwardDiff.derivative(f, x), …

---

## [Best practices for automatic generation of Jacobians](https://discourse.julialang.org/t/best-practices-for-automatic-generation-of-jacobians/68035)

<div class="topic-metadata">

**Author:** [@Gravlax](https://discourse.julialang.org/u/Gravlax)\
**Replies:** 7\
**Last updated:** [August 27, 2022, 9:33pm UTC](https://discourse.julialang.org/t/best-practices-for-automatic-generation-of-jacobians/68035 "2022-08-27T21:33:24Z")

</div>

Hi everyone, I would like to discuss the best practices for automatic generation of Jacobians when solving PDEs. I understand that the ‘way to go’ is to (1) define a residual function (in loop-style - pure Julia), (2) u…

[Previous page](https://discourse.julialang.org/tag/forwarddiff/363.md?match_all_tags=true&page=2&tags%5B%5D=forwarddiff)

[Next page](https://discourse.julialang.org/tag/forwarddiff/363.md?match_all_tags=true&page=4&tags%5B%5D=forwarddiff)
