JuMP model warm start using Ipopt

[Xuekui_wang]

I am solving a non-convex problem using Ipopt, and I want to warm start the same problem.
That not only needs to set start variable value usingset_start_value, but also needs to set duals and multipliers.
How can I access all duals and multipliers and input them to the model?

set_optimizer(model, Ipopt.Optimizer)
optimize!(model)

There is a function set_optimal_start_values In JuMP document, but it can not be used with Ipopt.

function set_optimal_start_values(model::Model)
    # Store a mapping of the variable primal solution
    variable_primal = Dict(x => value(x) for x in all_variables(model))
    # In the following, we loop through every constraint and store a mapping
    # from the constraint index to a tuple containing the primal and dual
    # solutions.
    constraint_solution = Dict()
    for (F, S) in list_of_constraint_types(model)
        # We add a try-catch here because some constraint types might not
        # support getting the primal or dual solution.
        try
            for ci in all_constraints(model, F, S)
                constraint_solution[ci] = (value(ci), dual(ci))
            end
        catch
            @info("Something went wrong getting $F-in-$S. Skipping")
        end
    end
    # Now we can loop through our cached solutions and set the starting values.
    for (x, primal_start) in variable_primal
        set_start_value(x, primal_start)
    end
    for (ci, (primal_start, dual_start)) in constraint_solution
        set_start_value(ci, primal_start)
        set_dual_start_value(ci, dual_start)
    end
    return
end

[odow]

It looks like setting the constraint primal isn’t supported by Ipopt yet. Just comment out that line. I also added a line to set the dual of the nonlinear constraints.

julia> using JuMP, Ipopt

julia> function set_optimal_start_values(model::Model)
           # Store a mapping of the variable primal solution
           variable_primal = Dict(x => value(x) for x in all_variables(model))
           # In the following, we loop through every constraint and store a mapping
           # from the constraint index to a tuple containing the primal and dual
           # solutions.
           constraint_solution = Dict()
           nlp_dual_start = nonlinear_dual_start_value(model)
           for (F, S) in list_of_constraint_types(model)
               # We add a try-catch here because some constraint types might not
               # support getting the primal or dual solution.
               try
                   for ci in all_constraints(model, F, S)
                       constraint_solution[ci] = (value(ci), dual(ci))
                   end
               catch
                   @info("Something went wrong getting $F-in-$S. Skipping")
               end
           end
           # Now we can loop through our cached solutions and set the starting values.
           for (x, primal_start) in variable_primal
               set_start_value(x, primal_start)
           end
           for (ci, (primal_start, dual_start)) in constraint_solution
               # set_start_value(ci, primal_start)
               set_dual_start_value(ci, dual_start)
           end
           set_nonlinear_dual_start_value(model, nlp_dual_start)
           return
       end
set_optimal_start_values (generic function with 1 method)

julia> model = Model(Ipopt.Optimizer)
A JuMP Model
Feasibility problem with:
Variables: 0
Model mode: AUTOMATIC
CachingOptimizer state: EMPTY_OPTIMIZER
Solver name: Ipopt

julia> @variable(model, x >= 0)
x

julia> @NLconstraint(model, x^2 <= 1)
x ^ 2.0 - 1.0 ≤ 0

julia> @objective(model, Max, x)
x

julia> optimize!(model)
This is Ipopt version 3.14.4, running with linear solver MUMPS 5.4.1.

Number of nonzeros in equality constraint Jacobian...:        0
Number of nonzeros in inequality constraint Jacobian.:        1
Number of nonzeros in Lagrangian Hessian.............:        1

Total number of variables............................:        1
                     variables with only lower bounds:        1
                variables with lower and upper bounds:        0
                     variables with only upper bounds:        0
Total number of equality constraints.................:        0
Total number of inequality constraints...............:        1
        inequality constraints with only lower bounds:        0
   inequality constraints with lower and upper bounds:        0
        inequality constraints with only upper bounds:        1

iter    objective    inf_pr   inf_du lg(mu)  ||d||  lg(rg) alpha_du alpha_pr  ls
   0  9.9999900e-03 0.00e+00 1.98e+00  -1.0 0.00e+00    -  0.00e+00 0.00e+00   0
   1  1.1773946e-01 0.00e+00 9.86e-01  -1.0 1.08e-01    -  5.58e-01 1.00e+00f  1
   2  3.7890821e+00 1.34e+01 2.47e+00  -1.7 3.67e+00    -  6.52e-02 1.00e+00f  1
   3  2.0163545e+00 3.07e+00 3.94e-01  -1.7 1.77e+00    -  1.00e+00 1.00e+00h  1
   4  1.2438830e+00 5.47e-01 5.70e-02  -1.7 7.72e-01    -  1.00e+00 1.00e+00h  1
   5  1.0080550e+00 1.62e-02 4.63e-02  -1.7 2.36e-01    -  1.00e+00 1.00e+00h  1
   6  9.9794206e-01 0.00e+00 4.09e-04  -2.5 3.52e-02    -  1.00e+00 1.00e+00h  1
   7  9.9984294e-01 0.00e+00 8.28e-06  -3.8 3.90e-03    -  1.00e+00 1.00e+00h  1
   8  9.9999813e-01 0.00e+00 4.51e-08  -5.7 3.14e-04    -  1.00e+00 1.00e+00h  1
   9  1.0000000e+00 0.00e+00 6.87e-12  -8.6 3.78e-06    -  1.00e+00 1.00e+00h  1

Number of Iterations....: 9

                                   (scaled)                 (unscaled)
Objective...............:  -1.0000000024889628e+00    1.0000000024889628e+00
Dual infeasibility......:   6.8741856104894100e-12    6.8741856104894100e-12
Constraint violation....:   0.0000000000000000e+00    0.0000000000000000e+00
Variable bound violation:   0.0000000000000000e+00    0.0000000000000000e+00
Complementarity.........:   2.5127966495526885e-09    2.5127966495526885e-09
Overall NLP error.......:   2.5127966495526885e-09    2.5127966495526885e-09


Number of objective function evaluations             = 10
Number of objective gradient evaluations             = 10
Number of equality constraint evaluations            = 0
Number of inequality constraint evaluations          = 10
Number of equality constraint Jacobian evaluations   = 0
Number of inequality constraint Jacobian evaluations = 10
Number of Lagrangian Hessian evaluations             = 9
Total seconds in IPOPT                               = 0.004

EXIT: Optimal Solution Found.

julia> set_optimal_start_values(model)

julia> optimize!(model)
This is Ipopt version 3.14.4, running with linear solver MUMPS 5.4.1.

Number of nonzeros in equality constraint Jacobian...:        0
Number of nonzeros in inequality constraint Jacobian.:        1
Number of nonzeros in Lagrangian Hessian.............:        1

Total number of variables............................:        1
                     variables with only lower bounds:        1
                variables with lower and upper bounds:        0
                     variables with only upper bounds:        0
Total number of equality constraints.................:        0
Total number of inequality constraints...............:        1
        inequality constraints with only lower bounds:        0
   inequality constraints with lower and upper bounds:        0
        inequality constraints with only upper bounds:        1

iter    objective    inf_pr   inf_du lg(mu)  ||d||  lg(rg) alpha_du alpha_pr  ls
   0  1.0000000e+00 0.00e+00 1.99e-09  -1.0 0.00e+00    -  0.00e+00 0.00e+00   0
   1  9.8764269e-01 0.00e+00 1.17e-02  -1.7 1.47e-02    -  1.00e+00 1.00e+00h  1
   2  9.9675597e-01 0.00e+00 4.71e-04  -2.5 1.82e-02    -  1.00e+00 1.00e+00h  1
   3  9.9983859e-01 0.00e+00 1.56e-05  -3.8 6.23e-03    -  1.00e+00 1.00e+00h  1
   4  9.9999813e-01 0.00e+00 4.55e-08  -5.7 3.29e-04    -  1.00e+00 1.00e+00h  1
   5  1.0000000e+00 0.00e+00 6.88e-12  -8.6 3.78e-06    -  1.00e+00 1.00e+00h  1

Number of Iterations....: 5

                                   (scaled)                 (unscaled)
Objective...............:  -1.0000000024889595e+00    1.0000000024889595e+00
Dual infeasibility......:   6.8763058686346261e-12    6.8763058686346261e-12
Constraint violation....:   0.0000000000000000e+00    0.0000000000000000e+00
Variable bound violation:   0.0000000000000000e+00    0.0000000000000000e+00
Complementarity.........:   2.5128010673512823e-09    2.5128010673512823e-09
Overall NLP error.......:   2.5128010673512823e-09    2.5128010673512823e-09


Number of objective function evaluations             = 6
Number of objective gradient evaluations             = 6
Number of equality constraint evaluations            = 0
Number of inequality constraint evaluations          = 6
Number of equality constraint Jacobian evaluations   = 0
Number of inequality constraint Jacobian evaluations = 6
Number of Lagrangian Hessian evaluations             = 5
Total seconds in IPOPT                               = 0.005

EXIT: Optimal Solution Found.

[Xuekui_wang]

Thank you @odow ， it is a great help for me.
Does it mean setting the constraint primals is not useful for Ipopt?
I solve your model by two steps: the first is to get all primal and dual values of variables and constraints.
Second, set all primal values and dual values, then set_optimizer(model, Ipopt.Optimizer) and optimize!(model).
The result is also 5 iterations.

function var_value(model::Model, constraint_solution::Dict, solution::Bool=true)
    for (F, S) in list_of_constraint_types(model)
        # We add a try-catch here because some constraint types might not
        # support getting the primal or dual solution.
        try
            if solution
                for ci in all_constraints(model, F, S)
                    constraint_solution[ci] = (value(ci), dual(ci))
                end
            else
                for ci in all_constraints(model, F, S)
                    constraint_solution[ci] =0.0
                end
            end
        catch
            @info("Something went wrong getting $F-in-$S. Skipping")
        end
    end
    return constraint_solution
end

variable_primal = Dict(string(x)=> value(x) for x in all_variables(model))
constraint_solution = Dict()
var_value(model, constraint_solution, true)
constraint_solution = Dict(string(x) =>(i,j) for (x,(i,j)) in constraint_solution)

model1 = Model()
@variable(model1, x>=0)
@NLconstraint(model1, x^2<=1)
@objective(model1, Max, x)

variable_start = Dict(x => start_value(x) for x in all_variables(model1))
constraint_solution_nopt = Dict()
var_value(model1, constraint_solution_nopt, false)

for (x, primal_start) in variable_start
    set_start_value(x, variable_primal[string(x)])
end
for (ci, x) in constraint_solution_nopt
    set_start_value(ci, constraint_solution[string(ci)][1])
    set_dual_start_value(ci, constraint_solution[string(ci)][2])
end

# nlp_dual_start = nonlinear_dual_start_value(model)
nlp_dual_start_opt = dual.(all_nonlinear_constraints(model))
set_nonlinear_dual_start_value(model1, nlp_dual_start_opt)

set_optimizer(model1, Ipopt.Optimizer)
optimize!(model1)

This is Ipopt version 3.14.4, running with linear solver MUMPS 5.4.1.

Number of nonzeros in equality constraint Jacobian...:        0
Number of nonzeros in inequality constraint Jacobian.:        1
Number of nonzeros in Lagrangian Hessian.............:        1

Total number of variables............................:        1
                     variables with only lower bounds:        1
                variables with lower and upper bounds:        0
                     variables with only upper bounds:        0
Total number of equality constraints.................:        0
Total number of inequality constraints...............:        1
        inequality constraints with only lower bounds:        0
   inequality constraints with lower and upper bounds:        0
        inequality constraints with only upper bounds:        1

iter    objective    inf_pr   inf_du lg(mu)  ||d||  lg(rg) alpha_du alpha_pr  ls
   0  1.0000000e+00 0.00e+00 1.99e-09  -1.0 0.00e+00    -  0.00e+00 0.00e+00   0
   1  9.8764269e-01 0.00e+00 1.17e-02  -1.7 1.47e-02    -  1.00e+00 1.00e+00h  1
   2  9.9675597e-01 0.00e+00 4.71e-04  -2.5 1.82e-02    -  1.00e+00 1.00e+00h  1
   3  9.9983859e-01 0.00e+00 1.56e-05  -3.8 6.23e-03    -  1.00e+00 1.00e+00h  1
   4  9.9999813e-01 0.00e+00 4.55e-08  -5.7 3.29e-04    -  1.00e+00 1.00e+00h  1
   5  1.0000000e+00 0.00e+00 6.88e-12  -8.6 3.78e-06    -  1.00e+00 1.00e+00h  1

Number of Iterations....: 5

                                   (scaled)                 (unscaled)
Objective...............:  -1.0000000024889595e+00    1.0000000024889595e+00
Dual infeasibility......:   6.8763058686346261e-12    6.8763058686346261e-12
Constraint violation....:   0.0000000000000000e+00    0.0000000000000000e+00
Variable bound violation:   0.0000000000000000e+00    0.0000000000000000e+00
Complementarity.........:   2.5128010673512823e-09    2.5128010673512823e-09
Overall NLP error.......:   2.5128010673512823e-09    2.5128010673512823e-09


Number of objective function evaluations             = 6
Number of objective gradient evaluations             = 6
Number of equality constraint evaluations            = 0
Number of inequality constraint evaluations          = 6
Number of equality constraint Jacobian evaluations   = 0
Number of inequality constraint Jacobian evaluations = 6
Number of Lagrangian Hessian evaluations             = 5
Total seconds in IPOPT                               = 0.077

EXIT: Optimal Solution Found.