How to undershand the Objective goes greater while itering?

Welcome, please read Please read: make it easier to help you
It is much easier to help you if we know what package you are are usin and what options.
It is also much preferred to post text copies of code rather than screenshots.


As a general rule though,
the objective can increase because most optimization algorithm only has some samples of what the loss function looks like.
Often just a single point and the gradient at that point (some times an approximation to the second order derivative also).
So it can use the logic:
“seems like the loss function decrease in this direction. Lets try checking its value at a new point (say) 1.5 units in that direction.”
then it will check a new point,
and either the pattern did contine for the next 1.5 units and the function value is lower.
or it didn’t and it is higher – maybe it would have continued only if it had taken a 0.5 unit step.

2 Likes