# Non-constant types in out-of-place ODE solve with multshooting?

**URL:** <https://discourse.julialang.org/t/non-constant-types-in-out-of-place-ode-solve-with-multshooting/126482>\
**Category:** New to Julia\
**Created:** [March 2, 2025, 11:41pm UTC](https://discourse.julialang.org/t/non-constant-types-in-out-of-place-ode-solve-with-multshooting/126482 "2025-03-02T23:41:05Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![cbilllingham1983](https://avatars.discourse-cdn.com/v4/letter/c/a88e57/32.png) [@cbilllingham1983](https://discourse.julialang.org/u/cbilllingham1983)\
**Post date:** [March 2, 2025, 11:41pm UTC](https://discourse.julialang.org/t/non-constant-types-in-out-of-place-ode-solve-with-multshooting/126482/1 "2025-03-02T23:41:05Z")

</div>

Hi, I’m trying to use multiple shooting for my analysis. I’ve been trying to reproduce the simple script shown of the Julia SciML documentation and I keep running into an error:  
ERROR: Detected non-constant types in an out-of-place ODE solve, i.e. for  
`du = f(u,p,t)` we see `typeof(du) !== typeof(u/t)`. This is not  
supported by OrdinaryDiffEq.jl’s solvers. Please either make `f`  
type-constant (i.e. typeof(du) === typeof(u/t)) or use the mutating  
in-place form `f(du,u,p,t)` (which is type-constant by construction).

Note that one common case for this is when computing with GPUs, using  
`Float32` for `u0` and `Float64` for `tspan`. To correct this, ensure  
that the element type of `tspan` matches the preferred compute type,  
for example `ODEProblem(f,0f0,(0f0,1f0))` for `Float32`-based time.

typeof(du) = Vector{ForwardDiff.Dual{ForwardDiff.Tag{ODEFunction{false, SciMLBase.AutoSpecialize, var"#53#54", LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT\_OBSERVED), Nothing, Nothing, Nothing, Nothing}, Float32}, Float32, 12}}I am running Julia version 1.10.8 on a 2024 MacBook with Apple Silicon. My script modifies the code from the documentation to ensure all floating point values are Float64.

I keep think this must a bug in either… 1.10.8 or on the Apple Silicon hardware… or my packages are misaligned. Any help would be appreciated.

using ComponentArrays  
using Lux  
using DiffEqFlux  
using Optimization  
using OptimizationPolyalgorithms  
using OrdinaryDiffEq  
using Plots  
using DiffEqFlux: group\_ranges  
using Random

rng = Random.default\_rng() #Xoshiro(123)

datasize = 30  
u0 = [2.0, 0.0]  
tmin = 0.0  
tmax = 5.0  
tspan = (0.0, 5.0)  
tsteps = range(tspan[1], tspan[2]; length = datasize)

function trueODEfunc!(du, u, p, t)  
true\_A = [-0.1 2.0 ; -2.0 -0.1]  
#du .= ((u .^3)’ \* true\_A)’  
du .= ((u.^3)‘true\_A)’  
#du .= true\_A \* (u.^3)  
end

function trueODEfunc2!(du, u, p, t)  
du[1] = -0.1 \* u[1]^3 - 2.0 \* u[2]^3  
du[2] = 2.0 \* u[1]^3 - 0.1 \* u[2]^3  
end  
prob\_trueode = ODEProblem(trueODEfunc!, u0, tspan)  
ode\_data = Array(solve(prob\_trueode, Tsit5(); saveat = tsteps))

#define the neural network  
nn = Lux.Chain(x → x .^3, Lux.Dense(2, 16, tanh), Lux.Dense(16, 2))  
p\_init, st = Lux.setup(rng, nn)

neuralode = NeuralODE(nn, tspan, Tsit5(); saveat = tsteps)  
prob\_node = ODEProblem((u, p, t) → nn(u, p, st)[1], u0, tspan, ComponentArray(p\_init))

function plot\_multiple\_shoot(plt, preds, group\_size)  
step = group\_size - 1  
ranges = group\_ranges(datasize, group\_size)

```
for (i, ig) in enumerate(range)
    plot!(plt, tsteps[rg], preds[i][1,:]; markershape = :circle, label = "Group $(i)")
end

```

end

anim = Plots.Animation()  
iter = 0  
callback = function (p, l, preds; doplot = true)  
display(l)  
global iter  
iter += 1  
if doplot && iter % 1 == 0  
#plot the original data  
plt = scatter(tsteps, ode\_data[1,:]; label = “Data”)  
#plot the different predictions for individual shoot  
plot\_multiple\_shoot(plt, preds, group\_size)  
frame(anim)  
display(plot(plt))  
end  
return false  
end

#define parameters for multiple multiple shooting  
group\_size = 3  
continuity\_term = 200

function loss\_function(data, pred)  
return sum(abs2, data - pred)  
end

ps = ComponentArray(p\_init)  
pd, pax = getdata(ps), getaxes(ps)

function loss\_multiple\_shooting(p)  
ps = ComponentArray(p, pax)  
return multiple\_shoot(  
ps,  
ode\_data,  
tsteps,  
prob\_node,  
loss\_function,  
Tsit5(),  
group\_size; continuity\_term  
)  
end

adtype = Optimization.AutoZygote()  
optf = Optimization.OptimizationFunction((x,p) → loss\_multiple\_shooting(x), adtype)  
optprob = Optimization.OptimizationProblem(optf, pd)  
res\_ms = Optimization.solve(optprob, PolyOpt(); callback = callback)  
gif(anim, “multiple\_shooting.gif”; fps = 15)

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [March 3, 2025, 12:19am UTC](https://discourse.julialang.org/t/non-constant-types-in-out-of-place-ode-solve-with-multshooting/126482/2 "2025-03-03T00:19:26Z")

</div>

> [@cbilllingham1983](#):
>
> ComponentArray(p\_init)

Make this `ComponentArray{Float64}(p_init)`? You may be mixing Float32 and Float64

---

<div class="post-metadata">

**Author:** ![cbilllingham1983](https://avatars.discourse-cdn.com/v4/letter/c/a88e57/32.png) [@cbilllingham1983](https://discourse.julialang.org/u/cbilllingham1983)\
**Post date:** [March 3, 2025, 12:30am UTC](https://discourse.julialang.org/t/non-constant-types-in-out-of-place-ode-solve-with-multshooting/126482/3 "2025-03-03T00:30:15Z")

</div>

No change. Could something be mismatched with my package versions?

 ![image](https://global.discourse-cdn.com/julialang/original/3X/9/3/93b9b8691f4b821e4b2c8d05ab25edfef8365b32.png)

I’m working with Raj Dandekar through his SciML bootcamp. I understand you were his PhD advisor. He told me he’s reproduced the error on his (x86) MacBook Air running Julia 1.6.7. He’s puzzled as well

---

<div class="post-metadata">

**Author:** ![cbilllingham1983](https://avatars.discourse-cdn.com/v4/letter/c/a88e57/32.png) [@cbilllingham1983](https://discourse.julialang.org/u/cbilllingham1983)\
**Post date:** [March 3, 2025, 12:48am UTC](https://discourse.julialang.org/t/non-constant-types-in-out-of-place-ode-solve-with-multshooting/126482/4 "2025-03-03T00:48:27Z")

</div>

This shot shows the code as posted online.

 ![image](https://global.discourse-cdn.com/julialang/original/3X/9/b/9bd720a5701e4c6da5a553c83e7fadca47b38049.jpeg)  
It’s mixing Float32 values in u0 and tspan but I would think the coefficients in true\_A would be read as Float64. I tried to correct for that by making everything Float64 in my code but… problem still occurs

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [March 3, 2025, 11:05am UTC](https://discourse.julialang.org/t/non-constant-types-in-out-of-place-ode-solve-with-multshooting/126482/5 "2025-03-03T11:05:19Z")

</div>

Show the code you ran, and use triple backticks so it’s easier to read.

---

<div class="post-metadata">

**Author:** ![cbilllingham1983](https://avatars.discourse-cdn.com/v4/letter/c/a88e57/32.png) [@cbilllingham1983](https://discourse.julialang.org/u/cbilllingham1983)\
**Post date:** [March 4, 2025, 12:37am UTC](https://discourse.julialang.org/t/non-constant-types-in-out-of-place-ode-solve-with-multshooting/126482/6 "2025-03-04T00:37:44Z")

</div>

Hi I think see what’s triggering the “non-constant types out-of-place error”.  
It happens if I define u0 and tspan as Float64 by default (as in u0 = [2.0, 0.0]…)

If I define u0 and tspan as Float32  
u0 = Float32[2.0, 0.0]  
and tspan = (0.0f0, 5.0f0)

the out-of-place ODE solve error isn’t thrown.

```julia
using ComponentArrays
using Lux
using DiffEqFlux
using Optimization
using OptimizationPolyalgorithms
using OrdinaryDiffEq
using Plots
using DiffEqFlux: group_ranges
using Random

rng = Random.default_rng() #Xoshiro(123)

datasize = 30
u0 = [2.0, 0.0] #defining u0 and tspan as Float64 by default
tspan = (0.0, 5.0)

#u0 = Float32[2.0, 0.0] #defining u0 and tspan as Float32
#tspan = (0.0f0, 5.0f0)
tsteps = range(tspan[1], tspan[2]; length = datasize)

function trueODEfunc!(du, u, p, t)
    true_A = [-0.1 2.0 ; -2.0 -0.1]
    du .= ((u.^3)'true_A)'
end

prob_trueode = ODEProblem(trueODEfunc!, u0, tspan)
ode_data = Array(solve(prob_trueode, Tsit5(); saveat = tsteps))

#define the neural network
nn = Lux.Chain(x -> x .^3, Lux.Dense(2, 16, tanh), Lux.Dense(16, 2))
p_init, st = Lux.setup(rng, nn)

neuralode = NeuralODE(nn, tspan, Tsit5(); saveat = tsteps)
prob_node = ODEProblem((u, p, t) -> nn(u, p, st)[1], u0, tspan, ComponentArray{Float64}(p_init))

function plot_multiple_shoot(plt, preds, group_size)
    step = group_size - 1
    ranges = group_ranges(datasize, group_size)

    for (i, ig) in enumerate(range)
        plot!(plt, tsteps[rg], preds[i][1,:]; markershape = :circle, label = "Group $(i)")
    end
end

anim = Plots.Animation()
iter = 0
callback = function (p, l, preds; doplot = true)
    display(l)
    global iter 
    iter += 1
    if doplot && iter % 1 == 0
        #plot the original data
        plt = scatter(tsteps, ode_data[1,:]; label = "Data")
        #plot the different predictions for individual shoot
        plot_multiple_shoot(plt, preds, group_size)
        frame(anim)
        display(plot(plt))
    end
    return false
end

#define parameters for multiple multiple shooting 
group_size = 3
continuity_term = 200

function loss_function(data, pred)
    return sum(abs2, data - pred)
end

ps = ComponentArray(p_init)
pd, pax = getdata(ps), getaxes(ps)

function loss_multiple_shooting(p)
    ps = ComponentArray(p, pax)
    return multiple_shoot(
                     ps,
                     ode_data,
                      tsteps,
                      prob_node,
                      loss_function,
                      Tsit5(),
                      group_size; continuity_term
    )
end

adtype = Optimization.AutoZygote()
optf = Optimization.OptimizationFunction((x,p) -> loss_multiple_shooting(x), adtype)
optprob = Optimization.OptimizationProblem(optf, pd)
res_ms = Optimization.solve(optprob, PolyOpt(); callback = callback)
gif(anim, "multiple_shooting.gif"; fps = 15)

```

However, that means the script runs up to the second to last line (89) where  
res\_ms = Optimization.solve(optprob, PolyOpt); callback = callback)  
Triggers:

 ![image](https://global.discourse-cdn.com/julialang/original/3X/0/e/0ecff3dd546145073c4c4673ad89f3265dd0384f.png)

I had though I could solve this problem by keeping all floating point values as float64.

So…  
(a) should declaring all floating point variables Float64 have triggered the non-constant types error?  
(b) Any advice for resolving the issue with PolyOpt()?

I appreciate any assistance

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [March 4, 2025, 12:58am UTC](https://discourse.julialang.org/t/non-constant-types-in-out-of-place-ode-solve-with-multshooting/126482/7 "2025-03-04T00:58:06Z")

</div>

That’s because you’re mixing Float64 and Float32 values, which will upconvert to Float64. Make everything Float64 or Float32 if you want to make things easier. Neural network parameters are Float32 by default unless you say otherwise.

---

<div class="post-metadata">

**Author:** ![cbilllingham1983](https://avatars.discourse-cdn.com/v4/letter/c/a88e57/32.png) [@cbilllingham1983](https://discourse.julialang.org/u/cbilllingham1983)\
**Post date:** [March 4, 2025, 1:24am UTC](https://discourse.julialang.org/t/non-constant-types-in-out-of-place-ode-solve-with-multshooting/126482/8 "2025-03-04T01:24:12Z")

</div>

I understand that mixing Float32 and Float64 leads to an error with the optimizer. The code on the SciML site explicitly showed u0 and tspan declared as Float32 with the coefficients in the trueODEfunc defaulting to Float64. l I attempted to make all floating point values Float64. This resulted in the “non-constant types” error.

On Raj’s advice I _also_ tried making all floating point values Float32:

```julia
using ComponentArrays
using Lux
using DiffEqFlux
using Optimization
using OptimizationPolyalgorithms
using OrdinaryDiffEq
using Plots
using DiffEqFlux: group_ranges
using Random

rng = Random.default_rng() #Xoshiro(123)

#Define initial conditions and time steps
datasize = 30
u0 = Float32[2.0,0.0]
tspan = (0.0f0, 5.0f0)
tsteps = range(tmin, tmax; length = datasize) #will be Float32

#Get the data
function trueODEfunc(du, u, p, t)
    #first = Float32(-0.1)
    #second = Float32(2.0)
    #third = Float32(-2.0)
    #fourth = Float32(-0.1)
    #true_A = [first second ; third fourth]
    true_A = Float32[-0.1 2.0; -2.0 -0.1]
    du .= ((u .^3)'true_A)'
    
end

prob_trueode = ODEProblem(trueODEfunc, u0, tspan)
ode_data = Array(solve(prob_trueode, Tsit5(); saveat = tsteps))

#define the neural network
nn = Lux.Chain(x -> x .^3, Lux.Dense(2 => 16, tanh), Lux.Dense(16 => 2))
#p_init, st = Lux.setup(rng, nn)
p64, st = Lux.setup(rng, nn)
p_init = Float32.(p64) #broadcast convert... THIS TRIGGERS AN ERROR

neuralode = NeuralODE(nn, tspan, Tsit5(); saveat = tsteps)
prob_node = ODEProblem((u, p, t) -> nn(u, p, st)[1], u0, tspan, ComponentArray(p_init))

function plot_multiple_shoot(plt, preds, group_size)
    step = group_size - 1
    ranges = group_ranges(datasize, group_size)

    for (i, ig) in enumerate(range)
        plot!(plt, tsteps[rg], preds[i][1,:]; markershape = :circle, label = "Group $(i)")
    end
end

anim = Plots.Animation()
iter = 0
callback = function (p, l, preds; doplot = true)
    display(l)
    global iter 
    iter += 1
    if doplot && iter % 1 == 0
        #plot the original data
        plt = scatter(tsteps, ode_data[1,:]; label = "Data")
        #plot the different predictions for individual shoot
        plot_multiple_shoot(plt, preds, group_size)
        frame(anim)
        display(plot(plt))
    end
    return false
end

#define parameters for multiple multiple shooting 
group_size = 3
continuity_term = 200

function loss_function(data, pred)
    return sum(abs2, data - pred)
end

ps = ComponentArray(p_init)
pd, pax = getdata(ps), getaxes(ps)

function loss_multiple_shooting(p)
    ps = ComponentArray(p, pax)
    return multiple_shoot(
                     ps,
                     ode_data,
                      tsteps,
                      prob_node,
                      loss_function,
                      Tsit5(),
                      group_size; continuity_term
    )
end

adtype = Optimization.AutoZygote()
optf = Optimization.OptimizationFunction((x,p) -> loss_multiple_shooting(x), adtype)
optprob = Optimization.OptimizationProblem(optf, pd)
res_ms = Optimization.solve(optprob, PolyOpt(); callback = callback)
gif(anim, "multiple_shooting.gif"; fps = 15)

```

This does NOT trigger the non-constant types error but it does throw an error when I try to cast p\_64 to Float32 as in:

 ![image](https://global.discourse-cdn.com/julialang/original/3X/a/b/ab149a98ed5ef8ae0e6e0765c0e163112ff76982.png)

Are these errors expected? I understand that all floating point values need to be consistently Float64 or Float32

---

<div class="post-metadata">

**Author:** ![abraemer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/abraemer/32/51403_2.png) [@abraemer](https://discourse.julialang.org/u/abraemer)\
**Post date:** [March 4, 2025, 2:54am UTC](https://discourse.julialang.org/t/non-constant-types-in-out-of-place-ode-solve-with-multshooting/126482/9 "2025-03-04T02:54:54Z")

</div>

Well the error message should be rather clear, is it not? You cannot convert the parameters to Float32 by doing `Float32.(p64)` because you cannot broadcast over a NamedTuple.

However, I think it might just be unnecessary as Chris said:

> [@ChrisRackauckas](#):
>
> Neural network parameters are Float32 by default unless you say otherwise.

So try removing this conversion.

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [March 4, 2025, 5:00am UTC](https://discourse.julialang.org/t/non-constant-types-in-out-of-place-ode-solve-with-multshooting/126482/10 "2025-03-04T05:00:30Z")

</div>

> [@cbilllingham1983](#):
>
> Are these errors expected? I understand that all floating point values need to be consistently Float64 or Float32

Since it’s nested named tuples, that’s the reason for the ComponentArray. `p_init = ComponentArray{Float32}(p64)` etc.