# Multiple shooting and multiple options

**URL:** <https://discourse.julialang.org/t/multiple-shooting-and-multiple-options/113659>\
**Category:** Modelling & Simulations\
**Created:** [April 30, 2024, 3:32pm UTC](https://discourse.julialang.org/t/multiple-shooting-and-multiple-options/113659 "2024-04-30T15:32:08Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![JordiBolibar](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jordibolibar/32/24307_2.png) [@JordiBolibar](https://discourse.julialang.org/u/JordiBolibar)\
**Post date:** [April 30, 2024, 3:32pm UTC](https://discourse.julialang.org/t/multiple-shooting-and-multiple-options/113659/1 "2024-04-30T15:32:08Z")

</div>

I am trying to add multiple shooting to a neural differential equation. After browsing multiple options, I see I have two possibilities:

- Use [DiffEqFlux.jl’s](https://docs.sciml.ai/DiffEqFlux/stable/examples/multiple_shooting/) `multiple_shoot`
- Use [DiffEqParamEstim.jl’](https://docs.sciml.ai/DiffEqParamEstim/stable/methods/optimization_based_methods/)s `multiple_shooting_objective`.

According to DiffEqFlux’s docs, this option “is a specialized form for implicit layer deep learning (known as data shooting) which assumes full observability of the underlying dynamics and lack of noise”.

I am unsure what is exactly meant by this. Right now, in our problem we are running an ODE forward as a ground truth and we are adding Fisher noise on top of it. Then, we are trying to learn the underlying dynamics from that “ground truth” using a neural ODE. Would DiffEqFlux be the way to go there, or it would only work for cases where the dynamics are perfectly clean and known? So far I have implemented this and it keeps going without even making it to the first epoch.

Would DiffEqParamEstim be a better alternative? I like the interface of `multiple_shoot` better, since it allows for more control.

If necessary, I can provide more details on the exact ODE and problem we are trying to solve. Thanks in advance!

---

<div class="post-metadata">

**Author:** ![Vaibhavdixit02](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/vaibhavdixit02/32/2916_2.png) [@Vaibhavdixit02](https://discourse.julialang.org/u/Vaibhavdixit02)\
**Post date:** [April 30, 2024, 3:35pm UTC](https://discourse.julialang.org/t/multiple-shooting-and-multiple-options/113659/2 "2024-04-30T15:35:15Z")

</div>

Prefer the DiffEqFlux one, that should be more recent

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [May 3, 2024, 12:26pm UTC](https://discourse.julialang.org/t/multiple-shooting-and-multiple-options/113659/3 "2024-05-03T12:26:45Z")

</div>

DiffEqFlux’s is a special form that I call data shooting. It’s not the more general form. I think someone should generalize it a bit before I’d recommend it more.

---

<div class="post-metadata">

**Author:** ![JordiBolibar](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jordibolibar/32/24307_2.png) [@JordiBolibar](https://discourse.julialang.org/u/JordiBolibar)\
**Post date:** [May 3, 2024, 12:45pm UTC](https://discourse.julialang.org/t/multiple-shooting-and-multiple-options/113659/4 "2024-05-03T12:45:30Z")

</div>

I see. What do you mean exactly by "it is a specialized form for implicit layer deep learning (known as data shooting) which assumes full observability of the underlying dynamics and lack of noise”? Do you need to have a dense continuous solution without noise for it to work?

We are trying it in our [SphereUDE](https://github.com/ODINN-SciML/SphereUDE.jl) project with @facusapienza and it doesn’t seem to converge for a controlled experiment with noise.

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [May 3, 2024, 12:47pm UTC](https://discourse.julialang.org/t/multiple-shooting-and-multiple-options/113659/5 "2024-05-03T12:47:17Z")

</div>

> [@JordiBolibar](#):
>
> I see. What do you mean exactly by "it is a specialized form for implicit layer deep learning (known as data shooting) which assumes full observability of the underlying dynamics and lack of noise”? Do you need to have a dense continuous solution without noise for it to work?

No I mean it takes the data points directly as samples for initial conditions, rather than having free floating initial conditions that are learned. This thus assumes the data points are “exactly correct” in some sense.

> [@JordiBolibar](#):
>
> We are trying it in our [SphereUDE](https://github.com/ODINN-SciML/SphereUDE.jl) project with @facusapienza and it doesn’t seem to converge for a controlled experiment with noise.

Not surprising.

---

<div class="post-metadata">

**Author:** ![JordiBolibar](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jordibolibar/32/24307_2.png) [@JordiBolibar](https://discourse.julialang.org/u/JordiBolibar)\
**Post date:** [May 3, 2024, 12:48pm UTC](https://discourse.julialang.org/t/multiple-shooting-and-multiple-options/113659/6 "2024-05-03T12:48:40Z")

</div>

OK, so what is the right tool for this? It’s really not clear in the docs.

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [May 6, 2024, 11:00am UTC](https://discourse.julialang.org/t/multiple-shooting-and-multiple-options/113659/7 "2024-05-06T11:00:00Z")

</div>

I don’t have an implementation I’d prefer right now.

---

<div class="post-metadata">

**Author:** ![facusapienza](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/facusapienza/32/24317_2.png) [@facusapienza](https://discourse.julialang.org/u/facusapienza)\
**Post date:** [May 8, 2024, 10:48pm UTC](https://discourse.julialang.org/t/multiple-shooting-and-multiple-options/113659/8 "2024-05-08T22:48:09Z")

</div>

@ChrisRackauckas I am thinking for this problem just to make an implementation of multiple shooting where the initial conditions for each batch can be learned. I understand the example in `multiple_shoot` is just for the noiseless case, so no initial condition is trained. I am planning to implement something like this in here [SphereUDE.jl](https://github.com/ODINN-SciML/SphereUDE.jl/blob/main/src/train.jl) by increasing the parameter to be optimized, in the same way we did this for the initial condition:

```julia
    # Set component vector for Optimization
    if params.train_initial_condition
        β = ComponentVector{Float64}(θ=θ, u0=params.u0)
    else
        β = ComponentVector{Float64}(θ=θ)
    end

```

I can try to work on an example that I can then PR to `DiffEqFlux.jl` if this is an useful contribution. I am happy to help here with an implementation for the case with noise, I just need some guidance of where to contribute and being sure this is a required feature.