# Variable time-stepping and discontinuous gradients

**URL:** <https://discourse.julialang.org/t/variable-time-stepping-and-discontinuous-gradients/91382>\
**Category:** Modelling & Simulations\
**Tags:** ode, optimization, differentialequation, ipopt\
**Created:** [December 7, 2022, 6:23pm UTC](https://discourse.julialang.org/t/variable-time-stepping-and-discontinuous-gradients/91382 "2022-12-07T18:23:39Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![nrontsis](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nrontsis/32/38535_2.png) [@nrontsis](https://discourse.julialang.org/u/nrontsis)\
**Post date:** [December 7, 2022, 6:23pm UTC](https://discourse.julialang.org/t/variable-time-stepping-and-discontinuous-gradients/91382/1 "2022-12-07T18:23:39Z")

</div>

I read [here](http://underactuated.mit.edu/trajopt.html) that in the context of “trajectory optimisation” or “optimal control”:

> One very important idea in numerical integration of differential equations is the use of variable-step integration … We typically avoid using variable steps inside a constraint (it can lead to discontinuous gradients)

What is your experience with optimising functions (with e.g. Ipopt) that entail variable-step integration? Is the resulting discontinuity in the gradients crippling to the performance of the optimisation solver?

Thanks!

---

<div class="post-metadata">

**Author:** ![nrontsis](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nrontsis/32/38535_2.png) [@nrontsis](https://discourse.julialang.org/u/nrontsis)\
**Post date:** [December 7, 2022, 6:47pm UTC](https://discourse.julialang.org/t/variable-time-stepping-and-discontinuous-gradients/91382/2 "2022-12-07T18:47:21Z")

</div>

[Related comment](https://news.ycombinator.com/item?id=29694886), but not sure how much it applies to variable-step differential equation solvers.

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [December 8, 2022, 7:38am UTC](https://discourse.julialang.org/t/variable-time-stepping-and-discontinuous-gradients/91382/3 "2022-12-08T07:38:50Z")

</div>

> [@nrontsis](#):
>
> What is your experience with optimising functions (with e.g. Ipopt) that entail variable-step integration? Is the resulting discontinuity in the gradients crippling to the performance of the optimisation solver?

If using continuous adjoint approaches, it does not lead to a discontinuity.

---

<div class="post-metadata">

**Author:** ![nrontsis](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nrontsis/32/38535_2.png) [@nrontsis](https://discourse.julialang.org/u/nrontsis)\
**Post date:** [December 8, 2022, 8:31am UTC](https://discourse.julialang.org/t/variable-time-stepping-and-discontinuous-gradients/91382/4 "2022-12-08T08:31:24Z")

</div>

That’s only when solving the differential equations to high precision, right?

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [December 8, 2022, 8:39am UTC](https://discourse.julialang.org/t/variable-time-stepping-and-discontinuous-gradients/91382/5 "2022-12-08T08:39:07Z")

</div>

No, because the derivative is a differential equation of a smooth function of `p` and `df/dp`, so if `f` is smooth then so its that derivative.

What is referred to is that adaptive ODE solvers’ solution is essentially noise below the tolerance limit. So if you solve at a tolerance of `1e-8`, then the solution can jump around by `1e-8` even for small changes in the parameters. This is because it does not guarantee any accuracy below that, and a change of a parameter can cause a step rejection which then gives a different solving process. However, that only is a problem if you’re using finite difference gradients: with continuous sensitivities the sensitivity equation is itself continuous and its calculation can be done with error controls.