ExaModels optimizer API for JuMP has convergence issue

Specific Domains Optimization (Mathematical)
1

It is my understanding that convergence should not depend on selection of AD package. I have tested many AD options on the same model and only one option has different convergence (number of iterations and reported values of each iteration) different from the others. That is Examodels’s experimental JuMP API: set_optimizer(pol, () -> ExaModels.IpoptOptimizer()).

I tested:

  1. default JuMP

  2. JuMP with symbolic AD

  3. JuMP to ASL via AmplNLWriter

  4. ExaModel(JuMP.Model())

  5. set_optimizer(pol, () → ExaModels.IpoptOptimizer())

  6. coding directly to ExaModels API

As said from above only option 5 differs from the others in terms of convergence. All option were run using Ipopt.jl, Ipopt_jll.jl or NLPModelsIpopt.jl .

I think this issue should be posted on ExaModels github, but it is possible that the error is somewhere else.

I attach the sample code. To run it download pglib_opf_case118_ieee.m from pglibopf and mwe.jl (4.5 KB)

The mwe implements options 1, 4 and 5. It uses JuMP, PowerModels, ExaModels, Ipopt, NLPModelsIpopt packages.

@sshin23 , @amontoison

2

Do you have the log of the solve (5)?

Does it find the same optimal solution? If so, this might be expected behavior, not a bug. You’re using a different optimizer, so the order of variables/constraints might be different.

3

Only 1 and 2 change the AD backend. 3, 4, 5, and 6 may have different orders for the variables and constraints. There is no strong guarantee that their iterates are identical.

4

Sure I have logs.

Log of solve (5).

This is Ipopt version 3.14.17, running with linear solver MUMPS 5.7.3.

Number of nonzeros in equality constraint Jacobian…: 6801
Number of nonzeros in inequality constraint Jacobian.: 1196
Number of nonzeros in Lagrangian Hessian…: 17108

Total number of variables…: 1088
variables with only lower bounds: 0
variables with lower and upper bounds: 0
variables with only upper bounds: 0
Total number of equality constraints…: 981
Total number of inequality constraints…: 824
inequality constraints with only lower bounds: 226
inequality constraints with lower and upper bounds: 0
inequality constraints with only upper bounds: 598

iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls
0 0.0000000e+00 2.77e+00 2.31e+01 -1.0 0.00e+00 - 0.00e+00 0.00e+00 0
1 8.1207678e+02 2.73e+00 2.27e+01 -1.0 1.32e+00 - 8.35e-03 1.45e-02h 1
2 2.7966034e+03 2.66e+00 2.27e+01 -1.0 1.04e+01 - 5.46e-03 2.39e-02h 1
3 1.0527326e+04 2.41e+00 2.36e+01 -1.0 9.64e+00 - 3.42e-02 9.43e-02h 1
4 1.4982943e+04 2.27e+00 1.89e+01 -1.0 7.99e+00 - 2.93e-01 6.13e-02h 1
5 3.2199961e+04 1.73e+00 1.44e+01 -1.0 7.68e+00 - 2.12e-01 2.36e-01h 1
6 4.5447086e+04 1.35e+00 1.12e+01 -1.0 6.56e+00 - 4.84e-01 2.22e-01h 1
7 6.0689844e+04 9.42e-01 2.25e+01 -1.0 6.81e+00 - 9.82e-01 3.00e-01h 1
8 7.9915948e+04 4.44e-01 9.92e+00 -1.0 7.98e+00 - 5.05e-01 5.29e-01h 1
9 8.6777586e+04 2.56e-01 8.24e+00 -1.0 1.44e+01 - 4.66e-01 4.23e-01h 1
iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls
10 8.7677775e+04 2.30e-01 9.82e+01 -1.0 1.86e+01 - 9.39e-01 1.01e-01h 1
11 9.3342442e+04 9.30e-02 5.33e+00 -1.0 1.52e+01 - 2.41e-01 5.96e-01h 1
12 9.4261345e+04 7.27e-02 9.42e+01 -1.0 4.32e+00 - 5.41e-01 2.18e-01h 1
13 9.7578950e+04 2.39e-02 1.34e+01 -1.0 2.07e+00 - 1.00e+00 9.72e-01h 1
14 9.7587577e+04 2.18e-02 1.19e+04 -1.0 6.89e+00 - 1.00e+00 1.23e-01h 1
15 9.7636335e+04 1.93e-02 9.58e+03 -1.0 6.14e+00 - 1.00e+00 6.48e-01h 1
16 9.7655154e+04 9.00e-03 2.23e+04 -1.0 3.94e+00 - 1.00e+00 5.80e-01h 1
17 9.7663706e+04 3.82e-03 5.09e+04 -1.0 1.78e+00 - 1.00e+00 5.90e-01h 1
18 9.7667276e+04 1.57e-03 1.19e+05 -1.0 7.56e-01 - 1.00e+00 5.95e-01h 1
19 9.7668765e+04 6.20e-04 2.68e+05 -1.0 3.12e-01 - 1.00e+00 6.08e-01h 1
iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls
20 9.7669379e+04 2.24e-04 5.43e+05 -1.0 1.23e-01 - 1.00e+00 6.40e-01h 1
21 9.7669630e+04 6.20e-05 8.11e+05 -1.0 4.46e-02 - 1.00e+00 7.24e-01h 1
22 9.7669721e+04 2.91e-06 1.93e+05 -1.0 1.23e-02 - 1.00e+00 9.53e-01h 1
23 9.7669725e+04 3.45e-11 1.00e-06 -1.0 5.78e-04 - 1.00e+00 1.00e+00f 1
24 9.7482887e+04 3.59e-03 3.17e+01 -8.6 5.07e+00 - 7.78e-01 5.55e-01f 1
25 9.7307258e+04 2.77e-03 2.74e+00 -8.6 2.45e+00 - 6.70e-01 7.24e-01f 1
26 9.7247210e+04 1.94e-03 1.25e+01 -8.6 3.12e+00 - 6.94e-01 6.12e-01h 1
27 9.7223396e+04 1.91e-03 7.70e+00 -8.6 1.75e+00 - 6.40e-01 6.18e-01h 1
28 9.7216576e+04 1.03e-03 3.14e+01 -8.6 6.01e-01 - 7.66e-01 5.56e-01h 1
29 9.7214726e+04 4.78e-04 3.02e+01 -8.6 1.93e-01 - 6.82e-01 5.39e-01h 1
iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls
30 9.7213811e+04 1.38e-04 3.14e+01 -8.6 8.08e-02 - 9.17e-01 7.13e-01h 1
31 9.7213603e+04 1.06e-05 1.30e+00 -8.6 1.80e-02 - 9.17e-01 9.45e-01h 1
32 9.7213603e+04 4.08e+02 3.37e+11 -8.6 1.50e+58 - 3.10e-62 1.36e-57f 1
33 9.7177579e+04 4.08e+02 3.37e+11 -8.6 5.16e+19 - 1.93e-10 4.49e-04F 1
34 9.6551411e+04 4.02e+02 3.34e+11 -8.6 2.44e+02 - 4.49e-04 7.81e-03f 1
35 9.6183976e+04 3.99e+02 3.33e+11 -8.6 2.39e+02 - 7.56e-03 4.77e-03f 1
36 9.6183967e+04 3.99e+02 3.33e+11 -8.6 1.00e+21 - 6.64e-05 1.11e-07F 1
37 9.5530590e+04 3.94e+02 3.30e+11 -8.6 2.37e+02 - 1.68e-02 8.68e-03f 1
38 9.5530586e+04 3.94e+02 3.30e+11 -8.6 4.72e+02 - 2.03e-04 4.82e-08h 1
39 9.5343252e+04 3.92e+02 3.29e+11 -8.6 4.50e+02 - 5.77e-06 2.59e-03f 1
iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls
40 9.5307681e+04 3.92e+02 3.29e+11 -8.6 4.09e+02 - 9.67e-04 5.02e-04f 1
In iteration 40, 1 Slack too small, adjusting variable bound
41 9.5307676e+04 3.92e+02 3.29e+11 -8.6 2.90e+02 - 6.15e-05 6.38e-08h 1
In iteration 41, 1 Slack too small, adjusting variable bound
42 9.5278536e+04 3.92e+02 3.29e+11 -8.6 3.27e+02 -4.0 3.11e-07 4.15e-04f 1
43r 9.5278536e+04 3.92e+02 1.00e+03 2.6 0.00e+00 - 0.00e+00 7.74e-60R 2
44r 9.5282579e+04 3.91e+02 1.00e+03 2.6 3.84e+05 - 2.53e-06 4.99e-06f 1
45r 9.5282214e+04 3.90e+02 9.99e+02 1.2 3.96e+05 - 6.48e-04 2.16e-05f 1
46r 9.5243262e+04 3.33e+02 9.99e+02 1.2 1.88e+05 - 4.15e-04 3.15e-04f 1
47 9.5242755e+04 3.33e+02 1.00e+02 -8.6 2.27e+02 - 1.21e-04 5.16e-05h 1
48 9.5246426e+04 3.33e+02 1.00e+02 -8.6 5.78e+02 - 3.81e-05 1.74e-04h 1
In iteration 48, 1 Slack too small, adjusting variable bound
49 9.5251586e+04 3.33e+02 1.00e+02 -8.6 6.90e+02 - 6.22e-04 2.04e-04h 1
iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls
50 9.5533309e+04 3.31e+02 9.95e+01 -8.6 1.33e+02 - 4.38e-05 4.29e-03h 1
51 9.5583356e+04 3.29e+02 9.91e+01 -8.6 3.05e+02 - 2.29e-03 4.11e-03h 1
52 9.5676837e+04 2.91e+02 9.34e+01 -8.6 3.34e+02 - 6.01e-03 5.81e-02h 1
53 9.5694217e+04 2.83e+02 9.21e+01 -8.6 2.78e+02 - 1.51e-01 1.35e-02h 1
54 9.5434752e+04 2.63e+02 8.89e+01 -8.6 2.91e+02 - 1.46e-02 3.49e-02h 1
55r 9.5434752e+04 2.63e+02 9.99e+02 2.5 0.00e+00 - 0.00e+00 1.81e-10R 2
56r 9.4430375e+04 2.61e+02 9.99e+02 2.5 2.99e+10 - 1.52e-11 5.59e-11f 1
57r 9.4369664e+04 2.58e+02 9.97e+02 1.1 2.98e+05 - 2.12e-03 1.34e-04f 1
58 9.4368984e+04 2.58e+02 1.00e+02 -8.6 2.98e+02 - 2.62e-03 3.47e-06h 1
59 9.4332710e+04 2.54e+02 9.94e+01 -8.6 2.88e+02 - 4.71e-02 6.33e-03h 1
iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls
60 9.4444081e+04 2.31e+02 9.48e+01 -8.6 2.61e+02 - 8.60e-03 4.58e-02h 1
61 9.4461441e+04 2.27e+02 9.40e+01 -8.6 2.26e+02 - 2.96e-01 8.97e-03h 1
62 9.4666496e+04 1.88e+02 8.57e+01 -8.6 2.19e+02 - 8.61e-02 8.81e-02h 1
63 9.5222408e+04 1.02e+02 6.37e+01 -8.6 1.61e+02 - 2.63e-01 2.57e-01h 1
64 9.5225101e+04 1.02e+02 6.36e+01 -8.6 3.19e+01 - 6.98e-01 1.56e-03h 1
65 9.5332684e+04 9.00e+01 5.99e+01 -8.6 3.15e+01 - 2.19e-01 5.77e-02h 1
66 9.5968091e+04 3.63e+01 3.90e+01 -8.6 1.79e+01 - 2.21e-01 3.49e-01h 1
67 9.6200485e+04 2.27e+01 3.16e+01 -8.6 4.23e+01 - 3.35e-01 1.91e-01h 1
68 9.6664112e+04 4.35e+00 1.69e+01 -8.6 4.79e+01 - 7.93e-01 4.65e-01h 1
69 9.7057474e+04 7.52e-01 4.69e+00 -8.6 4.42e+01 - 7.82e-01 7.22e-01h 1
iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls
70 9.7200005e+04 6.41e-02 4.00e-01 -8.6 1.91e+01 - 8.71e-01 9.15e-01h 1
71 9.7213190e+04 2.00e-03 1.25e-02 -8.6 2.20e+00 - 9.76e-01 9.69e-01h 1
72 9.7213606e+04 1.75e-07 1.70e-04 -8.6 7.28e-02 - 9.96e-01 1.00e+00h 1
73 9.7213606e+04 1.09e-08 4.94e-07 -8.6 9.86e-04 - 1.00e+00 1.00e+00h 1
74 9.7213606e+04 1.97e-09 8.93e-08 -8.6 4.20e-04 - 1.00e+00 1.00e+00h 1
75 9.7213606e+04 1.57e-10 7.11e-09 -8.6 1.19e-04 - 1.00e+00 1.00e+00h 1

Number of Iterations…: 75

                               (scaled)                 (unscaled)

Objective…: 7.8032096224085831e+02 9.7213605897951071e+04
Dual infeasibility…: 7.1127946117074342e-09 8.8612307713728492e-07
Constraint violation…: 1.5679235340626541e-10 1.5679235340626541e-10
Variable bound violation: 0.0000000000000000e+00 0.0000000000000000e+00
Complementarity…: 6.2446510802511211e-09 7.7796839821197416e-07
Overall NLP error…: 7.1127946117074342e-09 8.8612307713728492e-07

Number of objective function evaluations = 83
Number of objective gradient evaluations = 73
Number of equality constraint evaluations = 83
Number of inequality constraint evaluations = 83
Number of equality constraint Jacobian evaluations = 78
Number of inequality constraint Jacobian evaluations = 78
Number of Lagrangian Hessian evaluations = 75
Total seconds in IPOPT = 0.223

EXIT: Optimal Solution Found.

Log of solve (1) and de-facto any other option.

This is Ipopt version 3.14.17, running with linear solver MUMPS 5.7.3.

Number of nonzeros in equality constraint Jacobian…: 4622
Number of nonzeros in inequality constraint Jacobian.: 935
Number of nonzeros in Lagrangian Hessian…: 8198

Total number of variables…: 1088
variables with only lower bounds: 0
variables with lower and upper bounds: 0
variables with only upper bounds: 0
Total number of equality constraints…: 1016
Total number of inequality constraints…: 563
inequality constraints with only lower bounds: 0
inequality constraints with lower and upper bounds: 191
inequality constraints with only upper bounds: 372

iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls
0 0.0000000e+00 2.77e+00 1.67e+01 -1.0 0.00e+00 - 0.00e+00 0.00e+00 0
1 3.4102196e+04 2.01e+00 7.75e+01 -1.0 2.85e+00 - 3.60e-03 2.75e-01h 1
2 3.4396654e+04 1.89e+00 7.28e+01 -1.0 7.59e+00 - 6.40e-02 5.97e-02h 1
3 3.4273181e+04 1.83e+00 7.00e+01 -1.0 1.18e+01 - 2.65e-01 3.21e-02f 1
4 4.7923352e+04 1.37e+00 5.26e+01 -1.0 6.87e+00 - 3.42e-01 2.48e-01h 1
5 5.5468818e+04 1.14e+00 4.33e+01 -1.0 7.00e+00 - 4.36e-01 1.70e-01h 1
6 6.8686236e+04 7.62e-01 3.63e+01 -1.0 7.75e+00 - 6.89e-01 3.32e-01h 1
7 7.7086089e+04 5.31e-01 3.40e+01 -1.0 9.74e+00 - 9.91e-01 3.03e-01h 1
8 8.9595963e+04 1.99e-01 1.38e+01 -1.0 1.16e+01 - 1.00e+00 6.25e-01h 1
9 9.1042151e+04 1.49e-01 1.10e+01 -1.0 2.17e+01 - 5.17e-01 2.54e-01h 1
iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls
10 9.3827870e+04 8.27e-02 6.96e+00 -1.0 1.10e+01 - 1.00e+00 4.44e-01h 1
11 9.4967929e+04 5.76e-02 4.51e+00 -1.0 3.85e+00 - 6.22e-01 3.04e-01h 1
12 9.7658526e+04 2.88e-02 8.41e-01 -1.0 1.91e+00 - 1.00e+00 1.00e+00h 1
13 9.7652394e+04 3.39e-02 2.99e-01 -1.0 7.50e+00 - 1.00e+00 1.00e+00h 1
14 9.7670711e+04 2.37e-03 3.64e-03 -1.0 5.10e-01 - 1.00e+00 1.00e+00h 1
15 9.7484729e+04 4.77e-03 1.37e+00 -2.5 5.14e+00 - 8.01e-01 5.67e-01f 1
16 9.7309501e+04 2.78e-03 3.29e-01 -2.5 2.42e+00 - 7.58e-01 7.59e-01f 1
17 9.7248434e+04 1.93e-03 1.42e+00 -2.5 2.53e+00 - 1.00e+00 7.56e-01h 1
18 9.7231030e+04 9.94e-04 1.18e-02 -2.5 8.76e-01 - 1.00e+00 1.00e+00h 1
19 9.7232807e+04 2.71e-06 9.05e-05 -2.5 5.49e-02 - 1.00e+00 1.00e+00h 1
iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls
20 9.7219146e+04 2.35e-04 1.79e-01 -3.8 6.17e-01 - 8.60e-01 7.15e-01f 1
21 9.7214360e+04 6.77e-05 1.46e-01 -3.8 1.63e-01 - 8.71e-01 1.00e+00h 1
22 9.7214570e+04 4.67e-06 6.00e-05 -3.8 2.48e-02 - 1.00e+00 1.00e+00h 1
23 9.7213706e+04 6.33e-06 3.50e-03 -5.7 2.92e-02 - 9.21e-01 9.08e-01h 1
24 9.7213619e+04 7.26e-07 2.56e-05 -5.7 7.28e-03 - 1.00e+00 1.00e+00h 1
25 9.7213619e+04 1.13e-07 5.11e-06 -5.7 3.18e-03 - 1.00e+00 1.00e+00h 1
26 9.7213607e+04 4.77e-08 3.77e-05 -8.6 2.04e-03 - 9.88e-01 1.00e+00h 1
27 9.7213607e+04 1.05e-08 4.77e-07 -8.6 9.71e-04 - 1.00e+00 1.00e+00h 1
28 9.7213607e+04 1.93e-09 8.77e-08 -8.6 4.16e-04 - 1.00e+00 1.00e+00h 1
29 9.7213607e+04 1.52e-10 6.90e-09 -8.6 1.17e-04 - 1.00e+00 1.00e+00h 1

Number of Iterations…: 29

                               (scaled)                 (unscaled)

Objective…: 7.8032097059690670e+02 9.7213606938960656e+04
Dual infeasibility…: 6.9015999706667952e-09 8.5981211844802349e-07
Constraint violation…: 1.5211886958410581e-10 1.5211886958410581e-10
Variable bound violation: 0.0000000000000000e+00 0.0000000000000000e+00
Complementarity…: 6.1335000046158638e-09 7.6412102336905156e-07
Overall NLP error…: 6.9015999706667952e-09 8.5981211844802349e-07

Number of objective function evaluations = 30
Number of objective gradient evaluations = 30
Number of equality constraint evaluations = 30
Number of inequality constraint evaluations = 30
Number of equality constraint Jacobian evaluations = 30
Number of inequality constraint Jacobian evaluations = 30
Number of Lagrangian Hessian evaluations = 29
Total seconds in IPOPT = 1.311

EXIT: Optimal Solution Found.

Solve (5) does have the same solution but takes much more iterations to solve and runs into convergence issues (small slacks). Considering that all other options have basically identical iterates (not only the number of iterations, but also in every iteration the same numbers) for any power network I tested, I am convinced something is wrong here.

5

(5) has

Total number of variables…: 1088
variables with only lower bounds: 0
variables with lower and upper bounds: 0
variables with only upper bounds: 0
Total number of equality constraints…: 981
Total number of inequality constraints…: 824
inequality constraints with only lower bounds: 226
inequality constraints with lower and upper bounds: 0
inequality constraints with only upper bounds: 598

(1) has

Total number of variables…: 1088
variables with only lower bounds: 0
variables with lower and upper bounds: 0
variables with only upper bounds: 0
Total number of equality constraints…: 1016
Total number of inequality constraints…: 563
inequality constraints with only lower bounds: 0
inequality constraints with lower and upper bounds: 191
inequality constraints with only upper bounds: 372

They don’t appear to be solving the same model.

That is Examodels’s experimental JuMP AP

Open a bug report in ExaModels.jl. It looks like the “experimental” part is true.

1 Like
6

Opened bug report #130.