# Unable to compute gradient of loss of my model with sciml\_train

**URL:** <https://discourse.julialang.org/t/unable-to-compute-gradient-of-loss-of-my-model-with-sciml-train/49572>\
**Category:** General Usage\
**Tags:** question, sciml\
**Created:** [November 4, 2020, 9:45am UTC](https://discourse.julialang.org/t/unable-to-compute-gradient-of-loss-of-my-model-with-sciml-train/49572 "2020-11-04T09:45:21Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![TomCif](https://avatars.discourse-cdn.com/v4/letter/t/8e8cbc/32.png) [@TomCif](https://discourse.julialang.org/u/TomCif)\
**Post date:** [November 4, 2020, 9:45am UTC](https://discourse.julialang.org/t/unable-to-compute-gradient-of-loss-of-my-model-with-sciml-train/49572/1 "2020-11-04T09:45:21Z")

</div>

Hello,

I’m facing an error for which I can’t find any solution or reporting.

I’m trying to train a model defined by an ODE for which i solve the adjoint problem using concrete\_solve.

```julia
function initial_value_problem(du::AbstractArray, u::AbstractArray, p, t)
z = u[1:end-1]'
    f = re(q) 
    du[1:end-1] = f(z)
    du[end] = -t_J(f, z')
end

x = Float32.(rand(model, (1, 100)))
u0 = [Flux.flatten(x) 0]
prob = ODEProblem(initial_value_problem, u0, tspan)

function predict_adjoint(x, p) #we want to solve ivp with adjoint method 
    concrete_solve(prob,Tsit5(),u0 = x, p, #[u0,0f0]
                   saveat=0f0:0.1f0:10f0,sensealg=DiffEqFlux.InterpolatingAdjoint(
                   checkpointing=true))
end

function loss_adjoint(xs::AbstractArray, p)
    xs =[xs] 
    pz = Normal(0.0, 1.0)
    @showgrad preds = predict_adjoint(xs, p)[:,:,end]
    z = preds
    delta_logp = predict_adjoint(x, p)[:,:,end][end] 

    logpz = DistributionsAD.logpdf(pz, z)
    logpx = logpz .- delta_logp
    loss = -mean(logpx)
end

```

My loss function is a log-likelihood.  
I’m trying to use sciml\_train to train on some generated data but I always get the same error.

```julia
ERROR: MethodError: no method matching similar(::DiffEqBase.NullParameters)

```

Indeed when I try to manually use Zygote.gradient on my loss function it doesn’t work.  
Do you have any explanation? Is solve function compatible with Zygote.gradient?  
By the way my function t\_J is a function that I defined before computing trace of jacobian.

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [November 4, 2020, 10:43pm UTC](https://discourse.julialang.org/t/unable-to-compute-gradient-of-loss-of-my-model-with-sciml-train/49572/2 "2020-11-04T22:43:51Z")

</div>

> [@TomCif](#):
>
> ```julia
> function predict_adjoint(x, p) #we want to solve ivp with adjoint method 
> concrete_solve(prob,Tsit5(),u0 = x, p, #[u0,0f0]
> saveat=0f0:0.1f0:10f0,sensealg=DiffEqFlux.InterpolatingAdjoint(
> checkpointing=true))
> end
> 
> ```

I’m surprised that didn’t just error earlier. The issue is that you weren’t really passing `p` as it needed to be a keyword argument. That’s a deprecated function too, so the suggested updated syntax is:

```julia
function predict_adjoint(x, p) #we want to solve ivp with adjoint method
    _prob = remake(prob,u0=x,p=p)
          solve(_prob ,Tsit5(), #[u0,0f0]
                   saveat=0f0:0.1f0:10f0,sensealg=DiffEqFlux.InterpolatingAdjoint(
                   checkpointing=true))
end

```

---

<div class="post-metadata">

**Author:** ![ayaoyao214](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ayaoyao214/32/19370_2.png) [@ayaoyao214](https://discourse.julialang.org/u/ayaoyao214)\
**Post date:** [January 26, 2021, 3:16pm UTC](https://discourse.julialang.org/t/unable-to-compute-gradient-of-loss-of-my-model-with-sciml-train/49572/3 "2021-01-26T15:16:36Z")

</div>

Continuing the discussion from [Unable to compute gradient of loss of my model with sciml\_train](https://discourse.julialang.org/t/unable-to-compute-gradient-of-loss-of-my-model-with-sciml-train/49572/2):

Hi, I have met similar issue here. I defined a ODE problem with external input signal as the parameter. Here is my code. The wierd part is that, in my **loss\_batch** function, if the **loss l** is returned, I will have the error. If I test it by returning a constant value **l = 0.1** , the training will success.  
In this case, my thoughts is the **dudt** and **prediction\_batch** function should be fine, but something wrong with loss so that the gradient is unsupported. Any ideas about this? Thanks in advance.

```julia
nn_model = FastChain(FastDense(89,64, tanh), FastDense(64, 1))
pa = initial_params(nn_model)
u01 = Float32.([0.0])

function dudt(u,p,t)
    feature = p[1][Int(round(t*25+1)),:];
    nn_model(vcat(u, feature), p[2])
end

function predict_batch(fullp, x_input)
    prob_gpu2 = ODEProblem(dudt, u01, tspan,(x_input',fullp))
    Array(concrete_solve(prob_gpu2,Tsit5(),
    saveat = tsteps))
end

function loss_batch(fullp,x_input,y_output)
    pred =predict_batch(fullp,x_input)
    N = length(pred)
    l = sum(abs2, y_output[1:N] .- pred')
    #l = 0.1
    return l
end

res1 = DiffEqFlux.sciml_train(loss_batch,pa, ADAM(0.05), train_loader, maxiters =10)

```

 ![image](https://global.discourse-cdn.com/julialang/original/3X/c/5/c57472e1c0a9df22b23c19bbca80c9c069746cf5.png)

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [January 27, 2021, 2:22pm UTC](https://discourse.julialang.org/t/unable-to-compute-gradient-of-loss-of-my-model-with-sciml-train/49572/4 "2021-01-27T14:22:39Z")

</div>

train\_loader isn’t defined in your example.

---

<div class="post-metadata">

**Author:** ![ayaoyao214](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ayaoyao214/32/19370_2.png) [@ayaoyao214](https://discourse.julialang.org/u/ayaoyao214)\
**Post date:** [January 28, 2021, 6:32am UTC](https://discourse.julialang.org/t/unable-to-compute-gradient-of-loss-of-my-model-with-sciml-train/49572/5 "2021-01-28T06:32:36Z")

</div>

Hi thanks for replying.

train\_loader is defined via:

```julia
train_loader = DataLoader(xtrain, ytrain, batchsize=10, shuffle=true)

```

where x\_train is training features with feature dimensions of 88, and y\_train contains labels. In this toy example, I am using x\_train to be a 88\*300 matrix, and y\_label to be a 1 \* 300 array.

During debugging, I have checked the ODE solver works fine, I can print the solution of concreted\_solve every iteration, which means the forward path should be alright.

However, the error message seems to show there is a type mismatch in function where I defined ODEProblem with external input signal: `prob_gpu2 = ODEProblem(dudt, u01, tspan,(x_input',fullp))`, `(x_input',fullp)` is the only part I have used a Tuple.

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [January 28, 2021, 6:38am UTC](https://discourse.julialang.org/t/unable-to-compute-gradient-of-loss-of-my-model-with-sciml-train/49572/6 "2021-01-28T06:38:36Z")

</div>

Oh adjoints cannot have tuple parameters.