# How to calculate loss in a given time budget?

**URL:** <https://discourse.julialang.org/t/how-to-calculate-loss-in-a-given-time-budget/48369>\
**Category:** General Usage\
**Tags:** diffeq, sciml\
**Created:** [October 14, 2020, 2:36pm UTC](https://discourse.julialang.org/t/how-to-calculate-loss-in-a-given-time-budget/48369 "2020-10-14T14:36:53Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Alec\_4](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/alec_4/32/18466_2.png) [@Alec\_4](https://discourse.julialang.org/u/Alec_4)\
**Post date:** [October 14, 2020, 2:36pm UTC](https://discourse.julialang.org/t/how-to-calculate-loss-in-a-given-time-budget/48369/1 "2020-10-14T14:36:54Z")

</div>

Now i’d like to construct a Dataframe of loss values using different Optimizers  
in a **given processing time budget,** which should looks like below.  
But now my question is how to make this concrete?

│ Row │ Optimizer │ 1 min│ 2 mins │ 10 mins │ 20 mins │  
├───┼───────┼────┼─────┼─────┼──────┤  
│ 1 │ ADAM │ 1.2 │ 0.5 │ 0.32 │ 0.12 │  
│ 2 │ BFGS │ 1.3 │ 0.3 │ 0.26 │ 0.07 │  
│ 1 │ ADAMR │ 1.3 │ 0.4 │ 0.23 │ 0.12 │

---

<div class="post-metadata">

**Author:** ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)\
**Post date:** [October 14, 2020, 3:36pm UTC](https://discourse.julialang.org/t/how-to-calculate-loss-in-a-given-time-budget/48369/2 "2020-10-14T15:36:42Z")

</div>

I’m not sure what the question is: are you asking about the construction of a dataframe or about how to impose time constraints on different optimizers? If so, which packages are you using for fitting your models?

---

<div class="post-metadata">

**Author:** ![Alec\_4](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/alec_4/32/18466_2.png) [@Alec\_4](https://discourse.julialang.org/u/Alec_4)\
**Post date:** [October 14, 2020, 4:11pm UTC](https://discourse.julialang.org/t/how-to-calculate-loss-in-a-given-time-budget/48369/3 "2020-10-14T16:11:40Z")

</div>

Thanks for replying.  
I think there isn’t some special packages. I’m just using DiffeqFlux.sciml\_train to tune the parameters of my ODEProblem for reducing loss value.  
Yeah, the latter you said is right. Because now I want to use different optimizers like BFGS, ADAM etc to see how low can they get the loss function in a given time budget(1 min, 2mins, 10mins…). Do you have some idea about it?

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [October 14, 2020, 7:00pm UTC](https://discourse.julialang.org/t/how-to-calculate-loss-in-a-given-time-budget/48369/4 "2020-10-14T19:00:24Z")

</div>

You can just run a really long one, save in a callback the time for each few iterations, and then use that timeseries to get the values for the time to each error via rootfinding.

---

<div class="post-metadata">

**Author:** ![Alec\_4](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/alec_4/32/18466_2.png) [@Alec\_4](https://discourse.julialang.org/u/Alec_4)\
**Post date:** [October 15, 2020, 4:49pm UTC](https://discourse.julialang.org/t/how-to-calculate-loss-in-a-given-time-budget/48369/5 "2020-10-15T16:49:28Z")

</div>

Right, this makes sense. But could you please give me more instructions?  
Then how about making it in a function like form. But i can’t find good a way to  
make data transmission concrete.  
I can’t find a good way to exactly access my time budget. What i can do is  
only using a stupid loop like below because i don’t know if it’s possible to pass  
res\_10.time\_run attribute into callback as a judgment of whether to store the  
interested loss value.  
And all the code i’ve seen they just use normal callback in sciml\_train, is it possible to  
use other self-defined cb like VectorContinuousCallback() in sciml\_train?

```julia
@time for i in 1:50
    global res_10
    global time
     res_10 = DiffEqFlux.sciml_train(
        p -> bac_10(p; abstol=1e-2, reltol=1e-2),
        res_10.minimizer,
        DiffEqFlux.ADAM(0.5),
        maxiters = 1,
        cb = basic_bac_callback
        )
        time += res_10.time_run
end

```

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [October 15, 2020, 4:52pm UTC](https://discourse.julialang.org/t/how-to-calculate-loss-in-a-given-time-budget/48369/6 "2020-10-15T16:52:18Z")

</div>

Just do `time()` inside of a callback and save the array of times, then do `diff` of the result. Do it all in the callback to cutoff compile time outliers.

---

<div class="post-metadata">

**Author:** ![Alec\_4](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/alec_4/32/18466_2.png) [@Alec\_4](https://discourse.julialang.org/u/Alec_4)\
**Post date:** [October 18, 2020, 10:26am UTC](https://discourse.julialang.org/t/how-to-calculate-loss-in-a-given-time-budget/48369/7 "2020-10-18T10:26:29Z")

</div>

Thanks! Now i totally understand what you mean.