# Change loss function depending on the iteration count within Optim.jl

**URL:** <https://discourse.julialang.org/t/change-loss-function-depending-on-the-iteration-count-within-optim-jl/128362>\
**Category:** Optimization (Mathematical)\
**Tags:** optim\
**Created:** [April 24, 2025, 12:15pm UTC](https://discourse.julialang.org/t/change-loss-function-depending-on-the-iteration-count-within-optim-jl/128362 "2025-04-24T12:15:20Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![roflmaostc](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/roflmaostc/32/30123_2.png) [@roflmaostc](https://discourse.julialang.org/u/roflmaostc)\
**Post date:** [April 24, 2025, 12:15pm UTC](https://discourse.julialang.org/t/change-loss-function-depending-on-the-iteration-count-within-optim-jl/128362/1 "2025-04-24T12:15:20Z")

</div>

Hi,

let’s say I have a loss function which I want to change depending on the iteration count.

For example:

```julia
function l(x, target)
    target_warp = f_warp(target, iteration_count)
    sum(abs2, x .- target_warp)
end

```

Is there a way to do that? I am aware that the callback function has access to the iteration count so I could do some trick with a `Ref` but not sure how elegant this is.

Best,

Felix

PS: discourse reminded me that I joined this community exactly 5 years ago today haha

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [April 24, 2025, 10:53pm UTC](https://discourse.julialang.org/t/change-loss-function-depending-on-the-iteration-count-within-optim-jl/128362/2 "2025-04-24T22:53:51Z")

</div>

Hi @roflmaostc,

Why do you want to do this?

p.s., 🥳 🎂

---

<div class="post-metadata">

**Author:** ![roflmaostc](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/roflmaostc/32/30123_2.png) [@roflmaostc](https://discourse.julialang.org/u/roflmaostc)\
**Post date:** [April 25, 2025, 11:26am UTC](https://discourse.julialang.org/t/change-loss-function-depending-on-the-iteration-count-within-optim-jl/128362/3 "2025-04-25T11:26:26Z")

</div>

For [Gumbel Softmax](https://sassafras13.github.io/GumbelSoftmax/) there is a temperature parameter which is gradually reduced during training.

But in general, my loss function needs to be changed during training to push my solution towards a favorable state.

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [April 25, 2025, 10:21pm UTC](https://discourse.julialang.org/t/change-loss-function-depending-on-the-iteration-count-within-optim-jl/128362/4 "2025-04-25T22:21:17Z")

</div>

Whether this is a good idea depends on the algorithm you are using.

You could use your `Ref` trick, but note that some algorithms may not have a direct one-to-one relationship between “evaluating `l`” and “the iteration number”. For example, they may do a line search, or evaluate `l` at multiple points before taking a step.

---

<div class="post-metadata">

**Author:** ![roflmaostc](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/roflmaostc/32/30123_2.png) [@roflmaostc](https://discourse.julialang.org/u/roflmaostc)\
**Post date:** [April 25, 2025, 10:24pm UTC](https://discourse.julialang.org/t/change-loss-function-depending-on-the-iteration-count-within-optim-jl/128362/5 "2025-04-25T22:24:50Z")

</div>

I could use the callback which has access to the iteration number to change it.

But yes, calling loss is not directly the iteration number.