This is the iteration process for a problem, the result in iter 19 still acceptable.
In my understanding of optimization, this should not happen!
Is anybody know why waa that? I am really puzzled
Welcome, please read Please read: make it easier to help you
It is much easier to help you if we know what package you are are usin and what options.
It is also much preferred to post text copies of code rather than screenshots.
As a general rule though,
the objective can increase because most optimization algorithm only has some samples of what the loss function looks like.
Often just a single point and the gradient at that point (some times an approximation to the second order derivative also).
So it can use the logic:
“seems like the loss function decrease in this direction. Lets try checking its value at a new point (say) 1.5 units in that direction.”
then it will check a new point,
and either the pattern did contine for the next 1.5 units and the function value is lower.
or it didn’t and it is higher – maybe it would have continued only if it had taken a 0.5 unit step.
2 Likes
Also, Ipopt uses interior-point methods (hence its name) + line-search filter methods.
In non-convex optimization, the line-search does not apply directly to the objective function but rather to a merit function. Then, a step is accepted only if it leads to a sufficient decrease in the specified direction. So, in Ipopt, the evolution of the objective value along the iteration is not monotonous.
An example of merit function is for instance given in Equation (2.8), in this paper:
However, your problem is a bit awkward, as the objective goes from ~68.6 to 4,000 (and then to 3,000). Are you sure that your problem is smooth ? As stated by @oxinabox, a MWE would be of great help to debunk your problem.
1 Like
Thank you for your reply!
Actually, objective function include a differential process, and there are some failure situation which is not proper to calculate using the same function and I also can’t set constraint here. So I used some punish function here, and it is not continuous in some place.
Now the objective function keept in a proper palce for severial iters and then it came to a big value suddenly for a while, then back to the first proper place.
I am using Ipopt with solver mumps, are there any good ways to avoid this?
And this is my result with iter 100:
This is Ipopt version 3.12.10, running with linear solver mumps.
NOTE: Other linear solvers might be more efficient (see Ipopt documentation).
Number of nonzeros in equality constraint Jacobian…: 0
Number of nonzeros in inequality constraint Jacobian.: 0
Number of nonzeros in Lagrangian Hessian…: 0
Total number of variables…: 4
variables with only lower bounds: 0
variables with lower and upper bounds: 4
variables with only upper bounds: 0
Total number of equality constraints…: 0
Total number of inequality constraints…: 0
inequality constraints with only lower bounds: 0
inequality constraints with lower and upper bounds: 0
inequality constraints with only upper bounds: 0
iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls
0 2.9010486e+03 0.00e+00 6.25e+02 0.0 0.00e+00 - 0.00e+00 0.00e+00 0
1 3.1440370e+03 0.00e+00 3.75e+02 2.4 2.75e+02 - 6.65e-01 5.66e-03f 1
2 2.9880667e+03 0.00e+00 1.82e+02 2.1 5.59e-01 - 1.00e+00 1.00e+00f 1
3 2.9388803e+03 0.00e+00 1.31e+04 1.4 2.45e-01 - 9.11e-01 1.00e+00f 1
4 2.7704950e+03 0.00e+00 3.80e+04 1.4 2.55e+02 - 7.74e-03 1.36e-02f 2
5 1.1971720e+03 0.00e+00 1.62e+05 3.4 1.12e+03 - 1.66e-02 7.80e-04f 3
6 1.1971720e+03 0.00e+00 1.63e+05 3.0 1.12e+02 - 1.00e+00 2.09e-17f 51
7 1.1971720e+03 0.00e+00 1.63e+05 3.2 1.42e+02 - 5.67e-02 1.29e-19f 58
8 1.1971720e+03 0.00e+00 1.64e+05 2.5 4.00e+01 - 1.48e-01 5.71e-20f 61
9 8.8787005e+02 0.00e+00 3.43e+04 1.8 1.31e+03 - 2.80e-03 1.99e-03f 2
iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls
10 4.7725300e+02 0.00e+00 6.09e+03 3.0 1.14e-01 - 9.88e-01 1.00e+00f 1
11 9.7683633e+01 0.00e+00 3.43e+03 1.4 9.51e-02 - 1.00e+00 1.00e+00f 1
12 2.5424607e+01 0.00e+00 1.01e+03 -0.0 4.10e-02 - 1.00e+00 1.00e+00f 1
13 2.5420464e+01 0.00e+00 1.09e+03 -1.7 3.97e-02 - 1.00e+00 3.91e-03f 9
14 2.5420464e+01 0.00e+00 3.30e+02 -2.0 3.02e+00 - 1.00e+00 1.88e-13f 43
15 2.4792004e+01 0.00e+00 1.01e+03 -1.8 3.59e-03 - 1.00e+00 1.00e+00f 1
16 2.4791476e+01 0.00e+00 9.93e+02 -3.5 5.57e-03 - 1.00e+00 1.25e-01f 4
17 2.4791476e+01 0.00e+00 1.04e+03 -5.0 2.74e-02 - 1.00e+00 1.46e-11f 37
18 2.4791476e+01 0.00e+00 1.00e+03 -5.1 2.37e-03 - 1.00e+00 2.33e-10f 33
19 2.4791476e+01 0.00e+00 1.04e+03 -5.1 1.90e-05 - 1.00e+00 2.33e-10f 33
iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls
20 2.4791476e+01 0.00e+00 4.85e+01 -5.1 1.90e-05 - 1.00e+00 4.55e-13f 42
21 2.4791449e+01 0.00e+00 1.05e+03 -5.1 1.04e-06 - 1.00e+00 1.00e+00F 1
22 2.4790561e+01 0.00e+00 1.04e+03 -3.1 1.05e+03 - 1.94e-03 2.06e-08f 17
23 2.4789909e+01 0.00e+00 9.95e+02 -6.6 2.29e-05 - 1.00e+00 5.00e-01f 2
24 2.4789704e+01 0.00e+00 1.04e+03 -8.5 6.08e-05 - 1.00e+00 6.25e-02f 5
25 2.4789675e+01 0.00e+00 1.04e+03 -10.4 5.98e-06 - 1.00e+00 1.25e-01f 4
26 2.4788858e+01 0.00e+00 9.59e+02 -11.0 2.77e-05 - 1.00e+00 5.00e-01f 2
27 2.4788858e+01 0.00e+00 1.04e+03 -11.0 4.02e-06 - 1.00e+00 5.96e-08f 25
28 2.4788856e+01 0.00e+00 9.96e+02 -11.0 3.61e-06 - 1.00e+00 2.50e-01f 3
29 2.4788856e+01 0.00e+00 9.59e+02 -11.0 1.30e-05 - 1.00e+00 9.31e-10f 31
iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls
30 2.4788855e+01 0.00e+00 1.00e+03 -11.0 3.61e-06 - 1.00e+00 2.50e-01f 3
31 2.4788855e+01 0.00e+00 4.45e+01 -11.0 1.00e+03 - 1.61e-03 5.93e-18f 49
32 2.4782395e+01 0.00e+00 9.62e+02 -11.0 4.45e+01 - 3.86e-02 1.95e-06f 15
33 2.4782395e+01 0.00e+00 1.04e+03 -11.0 8.01e-05 - 1.00e+00 7.45e-09f 28
34 2.4782395e+01 0.00e+00 1.04e+03 -11.0 2.40e-05 - 1.00e+00 5.68e-14f 45
35 2.4782395e+01 0.00e+00 1.04e+03 -11.0 2.40e-05 - 1.00e+00 5.68e-14f 45
36 2.4783762e+01 0.00e+00 1.00e+03 -11.0 2.40e-05 - 1.00e+00 1.00e+00w 1
37 1.1048089e+03 0.00e+00 1.31e+04 -11.0 1.00e+03 - 6.33e-03 2.63e-03w 1
In iteration 37, 1 Slack too small, adjusting variable bound
38 1.1048089e+03 0.00e+00 2.27e+04 -11.0 6.90e+02 - 5.16e-02 3.82e-14w 1
39 2.4782395e+01 0.00e+00 1.04e+03 -11.0 2.41e+02 - 1.00e+00 5.68e-14f 44
iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls
40 2.4725235e+01 0.00e+00 8.49e+02 -11.0 4.59e+02 - 1.38e-02 4.49e-05f 8
41 2.4725235e+01 0.00e+00 8.49e+02 -8.9 8.49e+02 - 2.21e-03 3.24e-21f 60
42 2.4725235e+01 0.00e+00 8.49e+02 -10.9 8.49e+02 - 2.24e-03 3.27e-21f 60
43 2.4725235e+01 0.00e+00 8.49e+02 -10.9 8.49e+02 - 2.82e-03 3.27e-21f 60
44 2.4725235e+01 0.00e+00 8.49e+02 -10.9 8.49e+02 - 5.05e-03 3.27e-21f 60
45 2.4725235e+01 0.00e+00 8.49e+02 -10.9 8.49e+02 - 4.41e-03 3.27e-21f 60
46 2.4725235e+01 0.00e+00 8.49e+02 -10.9 8.49e+02 - 3.07e-03 3.27e-21f 60
47 2.4725235e+01 0.00e+00 8.49e+02 -10.9 8.49e+02 - 1.01e-02 3.27e-21f 60
48 2.4725235e+01 0.00e+00 8.49e+02 -10.9 8.49e+02 - 2.21e-03 3.27e-21f 60
49 2.4725235e+01 0.00e+00 8.49e+02 -10.9 8.49e+02 - 6.44e-01 3.27e-21f 60
iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls
50 2.4725235e+01 0.00e+00 8.49e+02 -10.9 8.49e+02 - 1.81e-03 3.27e-21f 60
51 2.4725235e+01 0.00e+00 8.49e+02 -10.9 8.49e+02 - 2.95e-03 3.27e-21f 60
52 4.0000000e+03 0.00e+00 5.00e+07 -10.9 8.49e+02 - 4.67e-03 1.89e-03w 1
53 4.0000000e+03 0.00e+00 5.00e+07 -10.9 4.92e+00 - 3.89e-12 6.38e-01w 1
54 3.1203239e+03 0.00e+00 3.38e+02 -10.9 1.57e+00 - 3.14e-11 1.00e+00w 1
55 4.0000000e+03 0.00e+00 5.00e+07 -10.9 1.06e-05 - 1.89e-03 1.89e-03S 12
56 3.1183745e+03 0.00e+00 3.15e+02 -10.9 1.33e+00 - 1.44e-11 1.00e+00f 1
57 3.1183719e+03 0.00e+00 3.15e+02 -10.9 8.35e-06 - 1.00e+00 1.00e+00f 1
58 3.1183693e+03 0.00e+00 3.15e+02 -10.9 8.35e-06 - 1.00e+00 1.00e+00f 1
59 3.1183666e+03 0.00e+00 3.15e+02 -10.9 8.35e-06 - 1.00e+00 1.00e+00f 1
iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls
60 2.7343984e+03 0.00e+00 2.39e+03 -10.9 3.15e+02 - 1.57e-04 2.46e-03f 3
61 2.6884001e+03 0.00e+00 4.38e+03 -10.9 2.39e+03 - 1.31e-03 4.85e-04f 2
62 2.6884001e+03 0.00e+00 1.74e+03 -10.9 4.38e+03 - 5.28e-04 2.94e-20f 54
63 2.1649391e+02 0.00e+00 4.46e+03 -10.9 1.74e+03 - 6.67e-04 2.85e-04f 3
64 7.8374439e+01 0.00e+00 2.96e+03 -10.9 4.46e+03 - 6.03e-04 2.08e-05f 5
65 2.6149164e+01 0.00e+00 4.01e+02 -10.9 3.57e-02 - 1.00e+00 1.00e+00f 1
66 2.5976194e+01 0.00e+00 5.51e+02 -10.9 4.38e-03 - 1.00e+00 5.00e-01f 2
67 2.5940039e+01 0.00e+00 5.33e+02 -10.9 9.06e-03 - 1.00e+00 2.50e-01f 3
68 2.5622437e+01 0.00e+00 5.34e+02 -10.9 1.62e-03 - 1.00e+00 1.00e+00f 1
69 2.5600510e+01 0.00e+00 5.44e+02 -10.9 5.23e-02 - 1.00e+00 3.12e-02f 6
iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls
70 2.5564467e+01 0.00e+00 5.37e+02 -10.9 5.45e-04 - 1.00e+00 1.00e+00f 1
71 2.5553100e+01 0.00e+00 5.40e+02 -10.9 2.12e-03 - 1.00e+00 5.00e-01f 2
72 2.5282679e+01 0.00e+00 3.58e+02 -10.9 1.56e-03 - 1.00e+00 1.00e+00f 1
73 2.5046210e+01 0.00e+00 8.41e+02 -10.9 9.36e-04 - 1.00e+00 1.00e+00f 1
74 2.4904612e+01 0.00e+00 5.59e+02 -10.9 1.08e-01 - 1.00e+00 1.56e-02f 7
75 2.4903474e+01 0.00e+00 5.50e+02 -10.9 1.37e-01 - 1.00e+00 9.77e-04f 11
76 2.4903474e+01 0.00e+00 5.50e+02 -10.9 2.03e-04 - 1.00e+00 1.42e-14f 47
77 2.4903474e+01 0.00e+00 5.50e+02 -10.9 2.03e-04 - 1.00e+00 1.42e-14f 47
78 2.4903474e+01 0.00e+00 5.50e+02 -10.9 2.03e-04 - 1.00e+00 1.42e-14f 47
79 2.4563135e+01 0.00e+00 6.85e+02 -10.9 5.50e+02 - 4.68e-03 1.50e-05f 10
iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls
80 2.4552262e+01 0.00e+00 6.46e+02 -11.0 3.56e-03 - 1.00e+00 5.00e-01f 2
81 2.4552256e+01 0.00e+00 6.15e+02 -11.0 1.08e-03 - 1.00e+00 4.88e-04f 12
82 2.4550513e+01 0.00e+00 6.63e+02 -11.0 1.05e-04 - 1.00e+00 2.50e-01f 3
83 2.4530383e+01 0.00e+00 6.09e+02 -11.0 4.06e-04 - 1.00e+00 1.00e+00f 1
84 2.4524251e+01 0.00e+00 6.15e+02 -11.0 6.75e-02 - 1.00e+00 1.56e-02f 7
85 2.4524003e+01 0.00e+00 5.99e+02 -11.0 4.23e-04 - 1.00e+00 6.25e-02f 5
86 2.4524003e+01 0.00e+00 6.15e+02 -11.0 4.45e-03 - 1.00e+00 1.53e-05f 17
87 2.4523860e+01 0.00e+00 5.99e+02 -11.0 1.96e-05 - 1.00e+00 5.00e-01f 2
88 2.4523860e+01 0.00e+00 5.91e+02 -11.0 7.14e-06 - 1.00e+00 1.49e-08f 27
89 2.4523860e+01 0.00e+00 5.74e+02 -11.0 8.49e-06 - 1.00e+00 2.98e-08f 26
iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls
90 2.4523860e+01 0.00e+00 5.74e+02 -11.0 2.17e-06 - 1.00e+00 7.28e-12f 38
91 2.4523860e+01 0.00e+00 5.74e+02 -11.0 2.17e-06 - 1.00e+00 7.28e-12f 38
92 2.4523860e+01 0.00e+00 5.74e+02 -11.0 2.17e-06 - 1.00e+00 7.28e-12f 38
93 2.4523860e+01 0.00e+00 5.74e+02 -11.0 5.74e+02 - 2.48e-03 8.30e-20f 56
94 2.4523860e+01 0.00e+00 5.74e+02 -11.0 5.74e+02 - 8.52e-02 2.08e-20f 58
95 2.4523860e+01 0.00e+00 5.74e+02 -11.0 5.74e+02 - 2.55e-03 2.08e-20f 58
96 2.4523860e+01 0.00e+00 5.74e+02 -11.0 5.74e+02 - 8.01e-01 2.08e-20f 58
97 2.4523860e+01 0.00e+00 5.74e+02 -11.0 5.74e+02 - 2.48e-03 2.08e-20f 58
98 2.4523860e+01 0.00e+00 5.74e+02 -11.0 5.74e+02 - 8.74e-02 2.08e-20f 58
99 4.5505774e+03 0.00e+00 1.51e+03 -11.0 5.74e+02 - 2.55e-03 2.99e-03w 1
iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls
100 4.5480459e+03 0.00e+00 1.52e+03 -11.0 6.73e-03 - 1.05e-08 1.00e+00w 1
As mentioned previously, it’s hard to offer advice without code.
Ipopt assumes that functions are differentiable, which doesn’t appear to be the case for your problem. So, the best way to avoid it is to reformulate your problem into a smooth, once (or twice) differentiable form (e.g., use a different penalty function).
You may want to read: https://projects.coin-or.org/CoinBinary/export/837/CoinAll/trunk/Installer/files/doc/Short%20tutorial%20Ipopt.pdf
I think I should choose another solver even the time should be longer.Great thanks to your advise again!