Ipopt infeasibility on travis-ci

Specific Domains Optimization (Mathematical)
1

Hi everyone.

I’m trying to set up tests for a project on Travis which uses JuMP with the Ipopt solver (repo for the project is here).

The build steps seem fine as far as I can tell and my tests can run (well, sometimes at least). However, the optimisation turns out to be infeasible in this environment, but not on my local machines.

Here is a link to a build log that’s infeasible, and another which just seems to spool forever.

Compare this with the output of the two problematic tests locally:

Total number of variables............................:     1561
                     variables with only lower bounds:      540
                variables with lower and upper bounds:      180
                     variables with only upper bounds:      120
Total number of equality constraints.................:     1391
Total number of inequality constraints...............:       59
        inequality constraints with only lower bounds:        0
   inequality constraints with lower and upper bounds:        0
        inequality constraints with only upper bounds:       59


Number of Iterations....: 142

                                   (scaled)                 (unscaled)
Objective...............:  -2.6702779245830361e+03   -2.6702779245830361e+03
Dual infeasibility......:   5.6843418860808015e-14    5.6843418860808015e-14
Constraint violation....:   3.9348429028973619e-11    3.9348429028973619e-11
Complementarity.........:   2.5059035671809955e-09    2.5059035671809955e-09
Overall NLP error.......:   2.5059035671809955e-09    2.5059035671809955e-09


Number of objective function evaluations             = 148
Number of objective gradient evaluations             = 108
Number of equality constraint evaluations            = 148
Number of inequality constraint evaluations          = 148
Number of equality constraint Jacobian evaluations   = 144
Number of inequality constraint Jacobian evaluations = 144
Number of Lagrangian Hessian evaluations             = 142
Total CPU secs in IPOPT (w/o function evaluations)   =     11.215
Total CPU secs in NLP function evaluations           =      1.649

EXIT: Optimal Solution Found.
Total number of variables............................:     1561
                     variables with only lower bounds:      540
                variables with lower and upper bounds:      180
                     variables with only upper bounds:      120
Total number of equality constraints.................:     1391
Total number of inequality constraints...............:       59
        inequality constraints with only lower bounds:        0
   inequality constraints with lower and upper bounds:        0
        inequality constraints with only upper bounds:       59


Number of Iterations....: 32

                                   (scaled)                 (unscaled)
Objective...............:  -2.6702779245830334e+03   -2.6702779245830334e+03
Dual infeasibility......:   5.6843418860808015e-14    5.6843418860808015e-14
Constraint violation....:   3.9348409947015384e-11    3.9348409947015384e-11
Complementarity.........:   2.5059035671809955e-09    2.5059035671809955e-09
Overall NLP error.......:   2.5059035671809955e-09    2.5059035671809955e-09


Number of objective function evaluations             = 33
Number of objective gradient evaluations             = 33
Number of equality constraint evaluations            = 33
Number of inequality constraint evaluations          = 33
Number of equality constraint Jacobian evaluations   = 33
Number of inequality constraint Jacobian evaluations = 33
Number of Lagrangian Hessian evaluations             = 32
Total CPU secs in IPOPT (w/o function evaluations)   =      1.817
Total CPU secs in NLP function evaluations           =      0.351

EXIT: Optimal Solution Found.
Total number of variables............................:     1561
                     variables with only lower bounds:      540
                variables with lower and upper bounds:      180
                     variables with only upper bounds:      120
Total number of equality constraints.................:     1391
Total number of inequality constraints...............:       59
        inequality constraints with only lower bounds:        0
   inequality constraints with lower and upper bounds:        0
        inequality constraints with only upper bounds:       59


Number of Iterations....: 32

                                   (scaled)                 (unscaled)
Objective...............:  -2.6702779245830334e+03   -2.6702779245830334e+03
Dual infeasibility......:   2.8421709430404007e-14    2.8421709430404007e-14
Constraint violation....:   3.9348409947015384e-11    3.9348409947015384e-11
Complementarity.........:   2.5059035671809955e-09    2.5059035671809955e-09
Overall NLP error.......:   2.5059035671809955e-09    2.5059035671809955e-09


Number of objective function evaluations             = 33
Number of objective gradient evaluations             = 33
Number of equality constraint evaluations            = 33
Number of inequality constraint evaluations          = 33
Number of equality constraint Jacobian evaluations   = 33
Number of inequality constraint Jacobian evaluations = 33
Number of Lagrangian Hessian evaluations             = 32
Total CPU secs in IPOPT (w/o function evaluations)   =      1.996
Total CPU secs in NLP function evaluations           =      0.393

EXIT: Optimal Solution Found.
Total number of variables............................:     1561
                     variables with only lower bounds:      540
                variables with lower and upper bounds:      180
                     variables with only upper bounds:       76
Total number of equality constraints.................:     1391
Total number of inequality constraints...............:       59
        inequality constraints with only lower bounds:        0
   inequality constraints with lower and upper bounds:        0
        inequality constraints with only upper bounds:       59


Number of Iterations....: 260

                                   (scaled)                 (unscaled)
Objective...............:  -2.6902447324365826e+03   -2.6902447324365826e+03
Dual infeasibility......:   5.6843418860808015e-14    5.6843418860808015e-14
Constraint violation....:   4.3934286764191199e-11    4.0218033681149734e-08
Complementarity.........:   2.5059035743924103e-09    2.5059035743924103e-09
Overall NLP error.......:   2.5059035743924103e-09    4.0218033681149734e-08


Number of objective function evaluations             = 294
Number of objective gradient evaluations             = 228
Number of equality constraint evaluations            = 294
Number of inequality constraint evaluations          = 294
Number of equality constraint Jacobian evaluations   = 263
Number of inequality constraint Jacobian evaluations = 263
Number of Lagrangian Hessian evaluations             = 260
Total CPU secs in IPOPT (w/o function evaluations)   =     20.503
Total CPU secs in NLP function evaluations           =      2.698

EXIT: Optimal Solution Found.
Total number of variables............................:     1561
                     variables with only lower bounds:      540
                variables with lower and upper bounds:      180
                     variables with only upper bounds:       76
Total number of equality constraints.................:     1391
Total number of inequality constraints...............:       59
        inequality constraints with only lower bounds:        0
   inequality constraints with lower and upper bounds:        0
        inequality constraints with only upper bounds:       59


Number of Iterations....: 32

                                   (scaled)                 (unscaled)
Objective...............:  -2.6902447128731592e+03   -2.6902447128731592e+03
Dual infeasibility......:   5.6843418860808015e-14    5.6843418860808015e-14
Constraint violation....:   4.3934286764191199e-11    4.3934286764191199e-11
Complementarity.........:   2.5059035743924124e-09    2.5059035743924124e-09
Overall NLP error.......:   2.5059035743924124e-09    2.5059035743924124e-09


Number of objective function evaluations             = 33
Number of objective gradient evaluations             = 33
Number of equality constraint evaluations            = 33
Number of inequality constraint evaluations          = 33
Number of equality constraint Jacobian evaluations   = 33
Number of inequality constraint Jacobian evaluations = 33
Number of Lagrangian Hessian evaluations             = 32
Total CPU secs in IPOPT (w/o function evaluations)   =      2.099
Total CPU secs in NLP function evaluations           =      0.317

EXIT: Optimal Solution Found.
Total number of variables............................:     1561
                     variables with only lower bounds:      540
                variables with lower and upper bounds:      180
                     variables with only upper bounds:       76
Total number of equality constraints.................:     1391
Total number of inequality constraints...............:       59
        inequality constraints with only lower bounds:        0
   inequality constraints with lower and upper bounds:        0
        inequality constraints with only upper bounds:       59


Number of Iterations....: 32

                                   (scaled)                 (unscaled)
Objective...............:  -2.6902447128731592e+03   -2.6902447128731592e+03
Dual infeasibility......:   2.8421709430404007e-14    2.8421709430404007e-14
Constraint violation....:   4.3934290233638151e-11    4.3934290233638151e-11
Complementarity.........:   2.5059035743924120e-09    2.5059035743924120e-09
Overall NLP error.......:   2.5059035743924120e-09    2.5059035743924120e-09


Number of objective function evaluations             = 33
Number of objective gradient evaluations             = 33
Number of equality constraint evaluations            = 33
Number of inequality constraint evaluations          = 33
Number of equality constraint Jacobian evaluations   = 33
Number of inequality constraint Jacobian evaluations = 33
Number of Lagrangian Hessian evaluations             = 32
Total CPU secs in IPOPT (w/o function evaluations)   =      2.080
Total CPU secs in NLP function evaluations           =      0.300

EXIT: Optimal Solution Found.
Test Summary: | Pass  Total
Utility       |    2      2
INFO: DICE tests passed

I’m using Arch Linux locally, and Travis uses Debian by default.

Does anyone know how I could start to troubleshoot this issue? It must have something to do with my environment setup on the Travis side, but without any specific error to chase down I’m a bit stuck.

2

This could be for any number of reasons. I don’t have any good way of troubleshooting this.

  • You may want to try providing a starting solution using the start keyword in @variable. (See the docs Nonlinear Modeling — JuMP -- Julia for Mathematical Optimization 0.18 documentation)

  • I would also try changing the tolerances in Ipopt.

  • If you have any random inputs make sure you set the same seed.

  • Your problem might also be poorly scaled. It looks like your objective value is on the order of 10^9. If you have very large and very small coefficients / variable values, it can lead to numeric difficulties with the solvers.

3

Thanks for the suggestions odow. I’ll have a fiddle with some of these ideas.

There are no random inputs so this can be scrubbed.

The other three points you make—whilst worth trying out—don’t seem to make sense to me as the culprit. All of them are based on the premise that my problem space is causing the issue, but as you can see in my successful tests: convergence happens quite rapidly, to the same optimal values with different input conditions and on multiple test machines.

To me therefore it’s most likely something to do with the environment, how Ipopt is built in the CI’s virtual machine vs my machines for example.

It’s not something imperative to solve, since I can just run the tests locally at any time. So I’ll give your suggestions a shot and if they don’t help then that’s fine for now as well.