Warm starting Ipopt with higher discretization points

[Dhormir]

Hello, I’m fairly new to Julia and using Ipopt as a solver, and currently I’m facing the following problem.

I’m solving an Optimal control problem with a discretization size of N. Then what I want to do is grab the provided solution and using it as an initial guess to the same problem but with a discretization size of N + m and keeping the same time-step size. What I’ve been trying to do is setting the values by using set_start_value on my problem variables using the previously found solution, but when I run Ipopt these initial values seem to get ignored which can be seen by the fact the initial objective value is not similar to the previous solution objective. From what I’ve read on the Ipopt documentation, it seems to show that there’s no expected improvement when there’s active set-changes between subsequent solutions:

If active-set changes are observed between subsequent solutions, then this strategy might not decrease the number of iterations (in some cases, it might even tend to increase the number of iterations.

But even then, one would expect to find similar initial objective values, right?
Any guidance or comments on this would be greatly appreciated, thanks.

[WalterMadelim]

You are likely to be asked to append an illustrating code example.

[odow]

Hi @Dhormir, welcome to the forum

Do you have a reproducible example? set_start_value should work to warm-start the primal solution. That paragraph you mentioned relates to warm-starting the dual multipliers.

If the objectives are different, then did you also warm-start the new variables also?

There’s a few things that could be happening, so it’s hard to say more without an example and the log from Ipopt.

[Dhormir]

Hi @WalterMadelim and @odow, thank you for the quick reply.

These are the logs I’m getting on Ipopt:

******************************************************************************
This program contains Ipopt, a library for large-scale nonlinear optimization.
 Ipopt is released as open source code under the Eclipse Public License (EPL).
         For more information visit https://github.com/coin-or/Ipopt
******************************************************************************

This is Ipopt version 3.14.17, running with linear solver MUMPS 5.7.3.

Number of nonzeros in equality constraint Jacobian...:   175976
Number of nonzeros in inequality constraint Jacobian.:        0
Number of nonzeros in Lagrangian Hessian.............:   644000

Total number of variables............................:    25000
                     variables with only lower bounds:     8000
                variables with lower and upper bounds:     1000
                     variables with only upper bounds:        0
Total number of equality constraints.................:    24000
Total number of inequality constraints...............:        0
        inequality constraints with only lower bounds:        0
   inequality constraints with lower and upper bounds:        0
        inequality constraints with only upper bounds:        0

iter    objective    inf_pr   inf_du lg(mu)  ||d||  lg(rg) alpha_du alpha_pr  ls
   0  1.0000000e-02 2.00e+01 2.00e+00   0.0 0.00e+00    -  0.00e+00 0.00e+00   0
   1  1.3990856e-02 1.75e+01 2.78e+03   0.3 2.00e+01    -  5.51e-04 1.25e-01f  1
   2  1.2644481e-01 1.74e+01 2.75e+03   1.3 2.80e+02    -  6.88e-04 3.51e-03f  1
   3  1.9293480e-01 1.74e+01 2.75e+03  -5.0 1.67e+03    -  2.37e-04 5.33e-04h  1
   4  3.4768228e-01 1.74e+01 2.74e+03   1.3 2.03e+04    -  3.70e-04 7.16e-04f  1
   5  4.7534659e-01 1.74e+01 2.74e+03   1.7 7.64e+04    -  5.33e-04 2.56e-04f  1
   6  7.8312629e-01 1.74e+01 2.73e+03   1.3 9.28e+04    -  4.14e-04 2.48e-04f  1
   7  1.2408953e+00 1.74e+01 2.73e+03  -5.0 1.02e+06  -2.0 7.44e-06 3.06e-05f  1
   8  1.8965625e+00 1.74e+01 2.73e+03   2.3 1.70e+05  -1.6 2.30e-05 1.30e-04f  1
   9  9.6716511e+00 1.73e+01 1.60e+03   1.9 2.85e+04  -1.1 6.01e-05 5.23e-03f  1
iter    objective    inf_pr   inf_du lg(mu)  ||d||  lg(rg) alpha_du alpha_pr  ls
  10  2.2008463e+01 1.72e+01 1.21e+03  -5.0 2.85e+04  -1.6 5.68e-04 3.66e-03f  1
  11  4.7167541e+01 1.72e+01 1.18e+03   1.9 7.84e+04  -2.1 1.60e-03 2.67e-03f  1
  12  5.5291706e+01 1.72e+01 1.20e+03   2.0 2.81e+05  -2.6 1.84e-03 2.36e-04f  1
  13  1.1819991e+02 1.70e+01 1.76e+03   2.0 4.09e+04  -2.2 6.36e-03 1.26e-02f  1
  14  5.2308610e+02 1.57e+01 3.63e+04   1.7 4.26e+04  -2.6 6.77e-02 7.77e-02f  1
  15  5.2233230e+02 1.57e+01 3.63e+04   3.2 2.51e+05    -  2.37e-03 6.43e-05f  1
  16  5.0976123e+02 1.56e+01 3.62e+04   3.2 1.35e+05    -  5.30e-03 1.55e-03f  1
  17  4.9987032e+02 1.56e+01 3.61e+04   3.4 1.05e+05    -  5.66e-03 2.37e-03f  1
  18  5.1271975e+02 1.55e+01 3.56e+04   3.6 4.68e+04    -  3.16e-03 8.85e-03f  1
  19  5.1033533e+02 1.55e+01 3.56e+04  -4.3 7.87e+04    -  7.30e-03 2.28e-04h  1
iter    objective    inf_pr   inf_du lg(mu)  ||d||  lg(rg) alpha_du alpha_pr  ls
  20  4.6922125e+02 1.53e+01 7.37e+04   3.1 5.88e+04    -  4.35e-01 6.93e-03h  1
  21  4.7760233e+02 1.53e+01 7.85e+04   3.8 8.29e+04    -  7.39e-03 1.26e-03f  1
  22  6.5224916e+02 2.30e+01 6.35e+04   3.9 8.36e+04    -  1.70e-02 2.85e-02f  1
  23  6.7125764e+02 2.29e+01 1.43e+05   3.9 3.99e+04    -  8.07e-02 2.96e-03f  1
  24  7.0184738e+02 2.22e+01 1.39e+05  -2.9 1.95e+04    -  5.32e-04 2.96e-02f  1
  25  7.1889699e+02 2.22e+01 1.73e+05   4.0 5.56e+04    -  2.68e-02 1.92e-03f  1
  26  7.5414461e+02 2.12e+01 1.65e+05  -2.9 1.23e+04    -  9.88e-04 4.69e-02f  1
  27  8.4996273e+02 2.10e+01 2.16e+05   4.0 5.54e+04    -  4.70e-02 1.04e-02f  1
  28  1.9975847e+03 5.00e+01 4.26e+05   3.8 3.50e+04    -  4.57e-01 1.89e-01f  1
  29  2.1890327e+03 4.93e+01 4.63e+05   3.2 2.84e+04    -  2.03e-01 2.82e-02f  1

Number of Iterations....: 90

                                   (scaled)                 (unscaled)
Objective...............:  -4.4714429076918845e+04    4.4714429076918845e+04
Dual infeasibility......:   1.7514581293531048e-12    1.7514581293531048e-12
Constraint violation....:   2.6261659513693303e-11    2.6261659513693303e-11
Variable bound violation:   9.9998062808734834e-13    9.9998062808734834e-13
Complementarity.........:   9.9792940057650538e-17    9.9792940057650538e-17
Overall NLP error.......:   2.6261659513693303e-11    2.6261659513693303e-11


Number of objective function evaluations             = 91
Number of objective gradient evaluations             = 91
Number of equality constraint evaluations            = 91
Number of inequality constraint evaluations          = 0
Number of equality constraint Jacobian evaluations   = 91
Number of inequality constraint Jacobian evaluations = 0
Number of Lagrangian Hessian evaluations             = 90
Total seconds in IPOPT                               = 25.903

EXIT: Solved To Acceptable Level.
This is Ipopt version 3.14.17, running with linear solver MUMPS 5.7.3.

Number of nonzeros in equality constraint Jacobian...:   180376
Number of nonzeros in inequality constraint Jacobian.:        0
Number of nonzeros in Lagrangian Hessian.............:   660100

Total number of variables............................:    25625
                     variables with only lower bounds:     8200
                variables with lower and upper bounds:     1025
                     variables with only upper bounds:        0
Total number of equality constraints.................:    24600
Total number of inequality constraints...............:        0
        inequality constraints with only lower bounds:        0
   inequality constraints with lower and upper bounds:        0
        inequality constraints with only upper bounds:        0

iter    objective    inf_pr   inf_du lg(mu)  ||d||  lg(rg) alpha_du alpha_pr  ls
   0  1.0000000e-02 2.19e+05 1.59e+04   0.0 0.00e+00    -  0.00e+00 0.00e+00   0
   1  4.5293316e+00 2.19e+05 1.59e+04   1.6 1.15e+06    -  2.34e-07 1.29e-04f  1
   2  2.4236770e+01 2.19e+05 1.59e+04  -0.1 2.25e+05    -  1.24e-04 5.32e-04f  1
   3  5.1823260e+01 2.19e+05 1.59e+04   0.7 2.16e+05    -  6.74e-04 7.10e-04f  1
   4  1.4188346e+02 2.18e+05 1.58e+04   0.7 1.25e+05  -4.0 1.92e-05 5.73e-03f  1
   5  1.0409508e+03 2.10e+05 1.53e+04   0.4 1.83e+05  -4.5 1.60e-03 3.32e-02f  1
   6  1.1472169e+03 2.09e+05 1.52e+04   0.4 1.60e+05  -5.0 1.08e-03 7.06e-03f  1
   7  1.5949910e+03 2.04e+05 1.48e+04   1.4 1.65e+05  -4.5 1.00e-02 2.26e-02f  1
   8  1.8297031e+03 2.02e+05 1.46e+04   1.4 1.59e+05  -5.0 2.91e-02 1.24e-02f  1
   9  2.2416280e+03 1.97e+05 1.43e+04   1.7 1.58e+05  -5.5 8.26e-01 2.10e-02f  1
iter    objective    inf_pr   inf_du lg(mu)  ||d||  lg(rg) alpha_du alpha_pr  ls
  10  4.3290108e+03 1.83e+05 1.33e+04   2.1 1.89e+05    -  2.81e-02 7.19e-02f  1
  11  4.4239359e+04 1.83e+03 2.62e+03   1.1 2.05e+05    -  4.48e-02 9.90e-01f  1
  12  4.5135417e+04 1.83e+01 3.34e+03  -4.4 4.73e+03    -  4.23e-01 9.90e-01f  1
  13  4.5818329e+04 1.73e+00 3.88e+05   0.1 5.38e+03    -  7.06e-01 9.90e-01f  1
  14  4.8631995e+04 4.46e+01 7.43e+07   0.8 4.59e+04    -  6.75e-01 5.67e-01f  1
  15  4.7642948e+04 9.34e+01 7.65e+08   1.3 6.41e+04    -  2.89e-01 1.47e-01f  1
  16  4.5334342e+04 1.11e+02 1.87e+08   0.8 9.25e+03    -  1.57e-01 4.35e-01f  1
  17  4.3468448e+04 8.51e+01 9.78e+08   0.8 1.19e+04    -  2.19e-01 4.41e-01f  1
  18  4.2072131e+04 6.23e+01 8.76e+08   0.8 1.28e+04    -  3.45e-01 3.84e-01f  1
  19  4.0495679e+04 4.72e+01 7.44e+08   0.8 1.42e+04    -  5.25e-01 4.16e-01f  1
iter    objective    inf_pr   inf_du lg(mu)  ||d||  lg(rg) alpha_du alpha_pr  ls
  20  3.7499882e+04 4.96e+01 7.13e+09   0.8 1.85e+04    -  1.00e+00 6.49e-01f  1
  21  3.4990458e+04 6.16e+01 9.89e+09   0.8 8.13e+04    -  2.04e-01 1.36e-01f  2
  22  3.3642314e+04 6.47e+01 1.02e+10   0.8 4.28e+05    -  2.23e-02 1.55e-02f  2
  23  3.1719061e+04 3.59e+01 1.95e+10   0.8 1.72e+04    -  1.00e+00 7.40e-01f  1
  24  3.3152358e+04 3.25e+00 2.88e-01   0.8 7.27e+03    -  1.00e+00 1.00e+00f  1
  25  3.6239928e+04 4.16e+01 3.15e+11   0.1 3.08e+04    -  2.31e-01 5.37e-01f  1
  26  3.8407105e+04 4.28e+01 1.39e+11   0.1 5.81e+04    -  3.29e-01 1.78e-01f  1
  27  4.1494687e+04 6.80e+01 8.58e+10   0.1 3.93e+04    -  3.34e-01 3.17e-01f  1
  28  4.3708908e+04 8.01e+01 4.71e+10   0.1 2.27e+04    -  3.20e-01 2.77e-01f  1
  29  4.5466042e+04 7.85e+01 1.68e+10   0.1 8.49e+03    -  4.14e-01 3.58e-01f  1

Number of Iterations....: 406

                                   (scaled)                 (unscaled)
Objective...............:  -5.0901903070405577e+04    5.0901903070405577e+04
Dual infeasibility......:   9.9999999998900724e-01    9.9999999998900724e-01
Constraint violation....:   2.5750068743946031e-11    2.5750068743946031e-11
Variable bound violation:   0.0000000000000000e+00    0.0000000000000000e+00
Complementarity.........:   3.9114264531644664e-11    3.9114264531644664e-11
Overall NLP error.......:   9.9999999998900724e-01    9.9999999998900724e-01


Number of objective function evaluations             = 2191
Number of objective gradient evaluations             = 381
Number of equality constraint evaluations            = 2191
Number of inequality constraint evaluations          = 0
Number of equality constraint Jacobian evaluations   = 410
Number of inequality constraint Jacobian evaluations = 0
Number of Lagrangian Hessian evaluations             = 407
Total seconds in IPOPT                               = 178.578

EXIT: Converged to a point of local infeasibility. Problem may be infeasible.

iteration: 1

time_steps: 1000
time_range: (0, 20)

Solution converged to a stationary point, but a local optimal solution is available

* Solver : Ipopt

* Status
  Result count       : 1
  Termination status : ALMOST_LOCALLY_SOLVED
  Message from the solver:
  "Solved_To_Acceptable_Level"

* Candidate solution (result #1)
  Primal status      : NEARLY_FEASIBLE_POINT
  Dual status        : NEARLY_FEASIBLE_POINT
  Objective value    : 4.47144e+04
  Dual objective value : -3.92920e+04

* Work counters
  Solve time (sec)   : 2.68816e+01
  Barrier iterations : 90

Objective value = 44714.429076918845


iteration: 2

time_steps: 1025
time_range: (0, 21)

Setting initial guess
old time_range: (0, 20)
new time_range: (0, 21)
The algorithm converged to an infeasible point or otherwise completed its search without finding a feasible solution, without guarantees that no feasible solution exists.
* Solver : Ipopt

* Status
  Result count       : 1
  Termination status : LOCALLY_INFEASIBLE
  Message from the solver:
  "Infeasible_Problem_Detected"

* Candidate solution (result #1)
  Primal status      : INFEASIBLE_POINT
  Dual status        : INFEASIBLE_POINT
  Objective value    : 5.09019e+04
  Dual objective value : -1.30396e-04

* Work counters
  Solve time (sec)   : 1.79809e+02
  Barrier iterations : 406

Objective value = 50901.90307040558

As you can see i expected the initial objective value on the second run to be closer in order of magnitude to the previously found objective.

What im using to set the start values on my new problem is this function:

function set_optimal_start_values(solved_model::GenericModel,
                                  unsolved_model::GenericModel)
    # In the following, we loop through every constraint and store a mapping
    # from the constraint index to a tuple containing the primal and dual
    # solutions.
    constraint_solved_solution = Dict()
    nlp_dual_start = nonlinear_dual_start_value(solved_model)
    for (F, S) in list_of_constraint_types(solved_model)
        # We add a try-catch here because some constraint types might not
        # support getting the primal or dual solution.
        try
            for ci in all_constraints(solved_model, F, S)
                constraint_solved_solution[string(ci)] = (value(ci), dual(ci))
            end
        catch
            @info("Something went wrong getting $F-in-$S. Skipping")
        end
    end
    constraint_unsolved_solution = Dict()
    for (F, S) in list_of_constraint_types(unsolved_model)
      # We add a try-catch here because some constraint types might not
      # support getting the primal or dual solution.
      try
          for ci in all_constraints(unsolved_model, F, S)
              constraint_unsolved_solution[string(ci)] = ci
          end
      catch
          @info("Something went wrong getting $F-in-$S. Skipping")
      end
    end
    # Store a mapping of the variable primal solution
    solved_variable_primal = Dict(string(x) => value(x) for x in all_variables(solved_model))
    unsolved_variable_primal = Dict(string(x) => x for x in all_variables(unsolved_model))
    # Now we can loop through our cached solutions and set the starting values.
    for (x, primal_start) in solved_variable_primal
      set_start_value(unsolved_variable_primal[x], primal_start)
    end

    errors = 0
    for (ci, (primal_start, dual_start)) in constraint_solved_solution
        try
            set_start_value(constraint_unsolved_solution[ci], primal_start)
        catch
            errors += 1
        end
        try
            set_dual_start_value(constraint_unsolved_solution[ci], dual_start)
        catch
            errors += 1
        end
    end
    @info( string(errors) * " errors found when setting constraints values")
    set_nonlinear_dual_start_value(unsolved_model, nlp_dual_start)
    set_optimizer_attribute(unsolved_model, "warm_start_init_point", "yes")
    return
end

I took some inspiration from the post JuMP model warm start using Ipopt (for some reason i cant post links). If theres anything else you might need just tell me.

[odow]

Do you have a reproducible example of the model?

I would try warm-starting only the primal solution. Leave off the dual starts.

Do all variables have unique names? Have you verified that these dictionaries are what you expect?

    solved_variable_primal = Dict(string(x) => value(x) for x in all_variables(solved_model))
    unsolved_variable_primal = Dict(string(x) => x for x in all_variables(unsolved_model))