# How does ForwardDiff.jl and FiniteDiff.jl handle edge-hitting cases?

**URL:** <https://discourse.julialang.org/t/how-does-forwarddiff-jl-and-finitediff-jl-handle-edge-hitting-cases/137828>\
**Category:** Optimization (Mathematical)\
**Tags:** question, package, forwarddiff, finitediff, autodiff\
**Created:** [June 27, 2026, 8:44pm UTC](https://discourse.julialang.org/t/how-does-forwarddiff-jl-and-finitediff-jl-handle-edge-hitting-cases/137828 "2026-06-27T20:44:38Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Xu\_Shan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xu_shan/32/214900_2.png) [@Xu\_Shan](https://discourse.julialang.org/u/Xu_Shan)\
**Post date:** [June 27, 2026, 8:44pm UTC](https://discourse.julialang.org/t/how-does-forwarddiff-jl-and-finitediff-jl-handle-edge-hitting-cases/137828/1 "2026-06-27T20:44:39Z")

</div>

Hi guys,

I am using ForwardDiff.jl to calculate the parameter uncertainty in my model (see below about the more detailed information…). I am wondering, how does `ForwardDiff.jl` and `FiniteDiff.jl` handle the edge-hitting parameters? i.e. what if the optimization finds one of the parameter is at the lower/upper bounds? and these bounds are defined in the model codes…

============  
detailed background:  
I am optimizing the parameters of a non-linear computer simulation of a physical process M(\theta, X), by attempting to find the simulational parameters \theta that make my simulation reproduce a set of summary metrics as similarly as possible to my observed values y\_0. To do this, I am using the following cost function:

L(\theta\_{optim}) = \min\_{\theta} L(y\_o, y\_{sim}(\theta)) =\sum\_{GPP, RECO,NEE,ET}\frac{(y\_o-y\_{sim}(\theta))^2}{var(y\_o)} + \sum\_{LAI,AGB}\frac{|y\_o-y\_{sim}(\theta)|}{1+mean(y\_o)} + \sum\_{NDVI}\frac{((y\_o-mean(y\_o))-(y\_{sim}-mean(y\_{sim}(\theta))))^2}{var(y\_o)}

where GPP, RECO, NEE, ET, LAI, AGB, NDVI are all different observation variables ( **time series** ). The idea is to jointly optimise across these different metrics, but weight their relative impact based on the measurement noise of the observed variables y\_0.

I intend to calculate the J matrix, where J is the Jacobian of the values of my simulated time series with regards to the underlying model parameters \theta, and would have the shape of `(n_time_series_point, n_parameter)` (Is this correct?). In other words, the matrix J would be the sensitivity of each data point in my observed timeseries data to the simulation parameters, at the optimal point in parameter space \theta\_{optim}.

I understand I can then use the matrix J^TJ to calculate the Hessian matrix, take the inverse of this Hessian, and use the square of the diagonal elements of the resulting matrix as a quantitative estimate of the uncertainty in my parameters given my data; as this has some precendent in the [literature](https://agupubs.onlinelibrary.wiley.com/doi/abs/10.1029/JC094iC05p06177).

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [June 27, 2026, 9:21pm UTC](https://discourse.julialang.org/t/how-does-forwarddiff-jl-and-finitediff-jl-handle-edge-hitting-cases/137828/2 "2026-06-27T21:21:13Z")

</div>

> [@Xu\_Shan](#):
>
> I am wondering, how does `ForwardDiff.jl` and `FiniteDiff.jl` handle the edge-hitting parameters?

First, ForwardDiff is an _analytical_ derivative (computed by “forward-mode” AD), and FiniteDiff is _approximate_ numerical derivative computed by finite differences. These are very different things!

Second, you need to be precise about what you mean by “edge-hitting” parameters. Suppose you have a differentiable function f(x), and you want to compute (or approximate) f'(x). By “edge-hitting”, I imagine you could mean one of two things.

- (a) You are optimizing f(x) in some domain, say x \in [a,b], but the function is still _defined_ outside that domain.
  - Then, both packages should work fine in computing f'(b) at the “edge”: ForwardDiff will compute the analytical derivative, using only computations at b, whereas FiniteDiff will look at points like b \pm \Delta x, so it may evaluate f slightly outside the domain… but so what?

- (b) Your function f(x) is _not defined_ outside the domain. In that case, the derivative f'(b) at the “edge” x=b is **not even defined** , since the conventional definition of the derivative requires the _two-sided limit_ to exist. In this case:
  - ForwardDiff will _probably_ return an answer corresponding to a one-sided limit (if it exists!), though it may pick the “wrong” side" — when there is a discontinuity in a function, it typically picks one side or the other depending on how the `if` statements are written.
  - FiniteDiff will default to a two-sided limit (via a centered difference) and fail or return nonsense, but you can pass ~~`ForwardMode` or `BackwardMode`~~ `fdtype=Val{:forward}` and `dir=1` or `dir=-1` to approximate one-sided limits from the right or left, assuming that’s the answer you want.

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [June 27, 2026, 10:04pm UTC](https://discourse.julialang.org/t/how-does-forwarddiff-jl-and-finitediff-jl-handle-edge-hitting-cases/137828/3 "2026-06-27T22:04:19Z")

</div>

> [@stevengj](#):
>
> - whereas FiniteDiff will look at points like b \pm \Delta x 𝑏 ±Δ𝑥, so it may evaluate f 𝑓 slightly outside the domain… but so what?

You can choose the differentiation direction, so you can tell it to perturb + or - depending on the right direction for the use case.

ForwardDiff uses partials which evaluate to `x + eps > x`, but if you use a negative `eps` (i.e. -(-f)'(x)) then you can similarly control the differentiation direction.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [June 27, 2026, 11:18pm UTC](https://discourse.julialang.org/t/how-does-forwarddiff-jl-and-finitediff-jl-handle-edge-hitting-cases/137828/4 "2026-06-27T23:18:02Z")

</div>

> [@ChrisRackauckas](#):
>
> You can choose the differentiation direction, so you can tell it to perturb + or - depending on the right direction for the use case.

Yes, that was my next bullet point. But this lowers the order of accuracy (compared to the default centered difference), so I would only do that if you actually _need_ a one-sided limit.

For example, to approximate \sin'(1) = \cos(1), these are the errors in the various finite differences with the default parameters:

```julia-auto
julia> FiniteDiff.finite_difference_derivative(sin, 1.0) - cos(1) # default centered difference
-5.036193684304635e-12

julia> FiniteDiff.finite_difference_derivative(sin, 1.0, Val{:forward}; dir=1) - cos(1) # forward difference, positive direction
-6.905069849238998e-9

julia> FiniteDiff.finite_difference_derivative(sin, 1.0, Val{:forward}; dir=-1) - cos(1) # forward difference, negative direction = backward difference
7.996091344608658e-9

```