# How often is Loss Function executed

**URL:** <https://discourse.julialang.org/t/how-often-is-loss-function-executed/60391>\
**Category:** New to Julia\
**Created:** [May 1, 2021, 5:57pm UTC](https://discourse.julialang.org/t/how-often-is-loss-function-executed/60391 "2021-05-01T17:57:43Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![hardy](https://avatars.discourse-cdn.com/v4/letter/h/c5a1d2/32.png) [@hardy](https://discourse.julialang.org/u/hardy)\
**Post date:** [May 1, 2021, 5:57pm UTC](https://discourse.julialang.org/t/how-often-is-loss-function-executed/60391/1 "2021-05-01T17:57:43Z")

</div>

I am beginner in Julia and try to understand the basics.  
I have an ODE problem with 2 parameters (PI control). These are to be learned from a measurement series.

My loss function looks like this:

```julia
function loss(p)             
     prediction = solve(prob_nn, Euler(), dt=Δt, p=p, saveat = t, sensealg=ForwardDiffSensitivity(convert_tspan=true))
     loss = sum(abs2, prediction[1:end-1,:].-X)     
     return loss, prediction
end

```

and it is called by sciml\_train:

```julia
result_ode = DiffEqFlux.sciml_train(loss, G_nn, BFGS()) #, cb = callback)

```

The algorithm finds the desired parameters in 8 iterations. Now I would like to know how often my loss function is actually called. I suspect that will happen more than 8 times (not just for calculating the gradients).  
How can I find out? I wanted to use Timeroutputs, but don’t know how. I can’t place the timer into the loss function like

```julia
@timeit to "loss_time" loss = sum(abs2, prediction[1:end-1,:].-X)

```

nor in sciml\_train.  
My next approach was the instrumenting profiler, but the IProfile package seems to be no longer available.  
I would be very grateful if someone could help me with this problem.

---

<div class="post-metadata">

**Author:** ![lostella](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lostella/32/356_2.png) [@lostella](https://discourse.julialang.org/u/lostella)\
**Post date:** [May 1, 2021, 6:25pm UTC](https://discourse.julialang.org/t/how-often-is-loss-function-executed/60391/2 "2021-05-01T18:25:43Z")

</div>

One general way I would do it is: define a callable structure, which holds a counter that gets increased any time it gets called.

For example you can wrap your loss in the following:

```julia
mutable struct Counting{T}
    f::T
    counter::Integer
end

(c::Counting)(x) = begin
    c.counter += 1
    c.f(x)
end

```

So instead of passing `loss` to whomever needs to call it, you pass `Counting(loss, 0)` and then go check the value of the counter at the end of the computation.

Edit: I interpreted your question as “how many times”, but I realized that by “how often” you mean to measure how much time passes between one call and the next. You can adapt the snippet above to record times using whatever timer mechanism, instead of counting.

---

<div class="post-metadata">

**Author:** ![bashonubuntu](https://avatars.discourse-cdn.com/v4/letter/b/f19dbf/32.png) [@bashonubuntu](https://discourse.julialang.org/u/bashonubuntu)\
**Post date:** [May 1, 2021, 7:02pm UTC](https://discourse.julialang.org/t/how-often-is-loss-function-executed/60391/3 "2021-05-01T19:02:25Z")

</div>

I don’t have the packages to run this on my machine. But, I think you can try

```julia
const to = TimerOutput();

@timeit to "loss_function" function loss(p)             
     prediction = solve(prob_nn, Euler(), dt=Δt, p=p, saveat = t, sensealg=ForwardDiffSensitivity(convert_tspan=true))
     loss = sum(abs2, prediction[1:end-1,:].-X)     
     return loss, prediction
end

result_ode = DiffEqFlux.sciml_train(loss, G_nn, BFGS())   

show(to)

```

---

<div class="post-metadata">

**Author:** ![czylabsonasa](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/czylabsonasa/32/8663_2.png) [@czylabsonasa](https://discourse.julialang.org/u/czylabsonasa)\
**Post date:** [May 1, 2021, 7:54pm UTC](https://discourse.julialang.org/t/how-often-is-loss-function-executed/60391/4 "2021-05-01T19:54:42Z")

</div>

another (ugly) way for counting the calls:

```julia
function mkfun()
  noc=0
  getnoc()=noc
  function fun(a,b)
    noc+=1
    a+b
  end
  fun,getnoc
end

f,fnoc=mkfun()
g,gnoc=mkfun()
h(a,b)=0
funarr=[f,g,h,f]
for k in 1:1000
  funarr[rand(1:length(funarr))](rand(1:10),rand(1:10))
end
println("f: ",fnoc()) # about a half
println("g: ",gnoc()) # about a quarter

```

drawback: you have to include the `noc+=1` into your function. ☹

If I were you, I would follow @lostella’s more general solution, which is - in my understanding - could be the following:

```julia
function myfun0()
  sleep(0.1*rand())
end

function myfun1(x)
  sleep(0.2*x)
end

function myfun2(x,y)
  sleep(x*y)
end

struct Cnt{F}
  f::F
  callsat::Array{Float64}
end

function (c::Cnt)(x...)
  push!(c.callsat,time())
  c.f(x...)
end

function caller(fun0,fun1,fun2,n)
  for k in 1:n
    r=rand()
    if r<1.0/3.0
      fun0()
    elseif r<2.0/3.0
      fun1(rand())
    else
      fun2(rand(),rand())
    end
  end
end  

cmyfun0=Cnt(myfun0,Float64[])
cmyfun1=Cnt(myfun1,Float64[])
cmyfun2=Cnt(myfun2,Float64[])

caller(cmyfun0,cmyfun1,cmyfun2,10)

[cmyfun0.callsat, cmyfun1.callsat, cmyfun2.callsat] .|> length |> println

```

Actually I never used `struct`s or `callable struct`s before, but they are very usable - worth to learn them!

---

<div class="post-metadata">

**Author:** ![yha](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yha/32/3502_2.png) [@yha](https://discourse.julialang.org/u/yha)\
**Post date:** [May 2, 2021, 1:49pm UTC](https://discourse.julialang.org/t/how-often-is-loss-function-executed/60391/5 "2021-05-02T13:49:53Z")

</div>

A functional style solution for creating a counter:

```julia
function counter(f)
    count = Ref(0)
    function (args...; kwargs...)
        count[] += 1
        f(args...; kwargs...)
    end, count
end

counting_loss, cnt = counter(loss)

```

But if this is just a one-time experiment to see what’s happening, it might be fine to simply increment a global

```julia
counter = 0
function loss(p)
  global counter += 1
  ...
end

```

---

<div class="post-metadata">

**Author:** ![hardy](https://avatars.discourse-cdn.com/v4/letter/h/c5a1d2/32.png) [@hardy](https://discourse.julialang.org/u/hardy)\
**Post date:** [May 4, 2021, 6:05pm UTC](https://discourse.julialang.org/t/how-often-is-loss-function-executed/60391/6 "2021-05-04T18:05:10Z")

</div>

Thanks for this idea. This is exactly what I was looking for. However, I did something wrong while trying it out and can’t find my mistake.  
I tried

```julia
struct Counting{T}
    f::T
    counter::Integer
end

(c::Counting)(x) = begin
    c.counter += 1
    c.f(x)
end

function testf(y)
y^2
end

cntr = Counting(testf, 0)

cntr(1.2)

```

But I can’t change the value of the counter. I get:  
ERROR: LoadError: setfield! immutable struct of type Counting cannot be changed

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [May 4, 2021, 6:15pm UTC](https://discourse.julialang.org/t/how-often-is-loss-function-executed/60391/7 "2021-05-04T18:15:37Z")

</div>

You want `mutable struct`

---

<div class="post-metadata">

**Author:** ![hardy](https://avatars.discourse-cdn.com/v4/letter/h/c5a1d2/32.png) [@hardy](https://discourse.julialang.org/u/hardy)\
**Post date:** [May 4, 2021, 6:25pm UTC](https://discourse.julialang.org/t/how-often-is-loss-function-executed/60391/8 "2021-05-04T18:25:39Z")

</div>

Hello Kristoffer,  
but why does czylabsonasa’s version (2. suggestion) work? It’s not a problem to concatenate to ‘callsat’.

---

<div class="post-metadata">

**Author:** ![lostella](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lostella/32/356_2.png) [@lostella](https://discourse.julialang.org/u/lostella)\
**Post date:** [May 4, 2021, 7:01pm UTC](https://discourse.julialang.org/t/how-often-is-loss-function-executed/60391/9 "2021-05-04T19:01:51Z")

</div>

yeah I fixed my snippet to add `mutable`, sorry about that, I didn’t really run it back then

---

<div class="post-metadata">

**Author:** ![hardy](https://avatars.discourse-cdn.com/v4/letter/h/c5a1d2/32.png) [@hardy](https://discourse.julialang.org/u/hardy)\
**Post date:** [May 4, 2021, 7:17pm UTC](https://discourse.julialang.org/t/how-often-is-loss-function-executed/60391/10 "2021-05-04T19:17:35Z")

</div>

Ah, then one can change an array in an immutable struct because the array itself is mutable (just a reference), but not a value? I think that I have even read that.

---

<div class="post-metadata">

**Author:** ![czylabsonasa](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/czylabsonasa/32/8663_2.png) [@czylabsonasa](https://discourse.julialang.org/u/czylabsonasa)\
**Post date:** [May 4, 2021, 7:26pm UTC](https://discourse.julialang.org/t/how-often-is-loss-function-executed/60391/11 "2021-05-04T19:26:40Z")

</div>

I think the answer is: if your intention is to reassign of some member of your struct then you should use mutable (for the primitive type Int the expression cntr+=1 is reassignment, we need mutable) but there is no reassignment for callsat in the above example, you only mutate its content. But maybe some expert will explain you better.  
read this: [Assignment and mutation - #4 by StefanKarpinski](https://discourse.julialang.org/t/assignment-and-mutation/19119/4)  
and this: [Julia Learning Circle: Memory Allocations and Garbage Collection · W.](https://wesselb.github.io/2020/11/23/julia-learning-circle-meeting-2.html)

---

<div class="post-metadata">

**Author:** ![hardy](https://avatars.discourse-cdn.com/v4/letter/h/c5a1d2/32.png) [@hardy](https://discourse.julialang.org/u/hardy)\
**Post date:** [May 5, 2021, 9:28am UTC](https://discourse.julialang.org/t/how-often-is-loss-function-executed/60391/12 "2021-05-05T09:28:03Z")

</div>

Thank you for this approach. I don’t want to use a global variable. But the first solution fits my purposes. Unfortunately, I can only mark one answer as a solution.