Powermodels.jl SCS for a mixed-integer problem with SDP relaxation

Hi all,

I am trying to solve a mixed integer problem of a network with switches (switches are my binary variables). I can solve my OPF problem with quadratic relaxations like ‘SOCWRPowerModel’, ‘QCRMPowerModel’, ‘QCLSPowerModel’. I would also like to solve my problem with SDP relaxation due to smaller optimality gaps. For that reason, I have set my solver as following with SCS to solve SDP:

mip_solver = JuMP.optimizer_with_attributes(
    Cbc.Optimizer,
    ...
)
nl_solver = JuMP.optimizer_with_attributes(
    SCS.Optimizer,
    ...
)
solver = JuMP.optimizer_with_attributes(
    Juniper.Optimizer,
    "nl_solver" => nl_solver,
    "mip_solver" => mip_solver,
)

I get the following output:

Status: Solved/Inaccurate
Hit max_iters, solution may be inaccurate, returning best found solution.
Timing: Solve time: 1.22e+001s
	Lin-sys: avg # CG iterations: 8.60, avg solve time: 1.17e-004s
	Cones: avg projection time: 2.03e-003s
	Acceleration: avg step time: 2.63e-004s
----------------------------------------------------------------------------
Error metrics:
dist(s, K) = 4.4009e-003, dist(y, K*) = 1.6233e-009, s'y/|s||y| = 9.9745e-016
primal res: |Ax + s - b|_2 / (1 + |b|_2) = 4.1278e-004
dual res:   |A'y + c|_2 / (1 + |c|_2) = 7.7015e-004
rel gap:    |c'x + b'y| / (1 + |c'x| + |b'y|) = 4.9388e-003
----------------------------------------------------------------------------
c'x = -36926.4630, -b'y = -37293.0217
============================================================================
┌ Warning: MathOptInterface.VariablePrimalStart() is not supported by SCS.GeometricConicForm{Float64, SCS.SparseMatrixCSRtoCSC{Int64}, NTuple{8, DataType}}. This information will be discarded.
└ @ MathOptInterface.Utilities C:\Users\XXX\.julia\packages\MathOptInterface\YDdD3\src\Utilities\copy.jl:290
┌ Warning: MathOptInterface.VariablePrimalStart() is not supported by SCS.GeometricConicForm{Float64, SCS.SparseMatrixCSRtoCSC{Int64}, NTuple{8, DataType}}. This information will be discarded.
└ @ MathOptInterface.Utilities C:\Users\XXX\.julia\packages\MathOptInterface\YDdD3\src\Utilities\copy.jl:290
┌ Warning: The relaxation is only almost solved.
└ @ Juniper C:\Users\XXX\.julia\packages\Juniper\8wso7\src\model.jl:78
MathOptInterface.UnsupportedConstraint{MathOptInterface.VectorAffineFunction{Float64}, MathOptInterface.SecondOrderCone}: `MathOptInterface.VectorAffineFunction{Float64}`-in-`MathOptInterface.SecondOrderCone` constraint is not supported by the model.

Stacktrace:
  [1] bridge_type(b::MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Cbc.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.GenericModel{Float64, MathOptInterface.Utilities.ModelFunctionConstraints{Float64}}}}}, F::Type{MathOptInterface.VectorAffineFunction{Float64}}, S::Type{MathOptInterface.SecondOrderCone})
    @ MathOptInterface.Bridges C:\Users\XXX\.julia\packages\MathOptInterface\YDdD3\src\Bridges\lazy_bridge_optimizer.jl:441
  [2] concrete_bridge_type(b::MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Cbc.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.GenericModel{Float64, MathOptInterface.Utilities.ModelFunctionConstraints{Float64}}}}}, F::Type{MathOptInterface.VectorAffineFunction{Float64}}, S::Type{MathOptInterface.SecondOrderCone})
    @ MathOptInterface.Bridges.Constraint C:\Users\XXX\.julia\packages\MathOptInterface\YDdD3\src\Bridges\Constraint\bridge.jl:151
  [3] add_constraint(b::MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Cbc.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.GenericModel{Float64, MathOptInterface.Utilities.ModelFunctionConstraints{Float64}}}}}, f::MathOptInterface.VectorAffineFunction{Float64}, s::MathOptInterface.SecondOrderCone)
    @ MathOptInterface.Bridges C:\Users\XXX\.julia\packages\MathOptInterface\YDdD3\src\Bridges\bridge_optimizer.jl:1480
...

I have used SCS also for an OPF problem without integer variables. The optimization could be solved in that case without any issue.

I have also tried with Mosek as nl_solver and I got the error as in this issue

@ccoffrin, Do you have any suggestion regarding solver setup for SDP relaxation for MIP problems? What might be the reason of the errors above?

I appreciate your response.
All the best

In addition, the versions of my packages:
JuMP: v0.21.10
PowerModels: v0.18.4
Juniper: v0.7.0
SCS: v0.7.1
Cbc: v0.8.1
MosekTools: v0.9.4

This is definitely fixed in the most-recent release of Juniper, but you’ll need PowerModels to be updated before you can use it.

For now, you could try updating your packages and see if it helps import Pkg; Pkg.update(). You’re a few steps behind so not sure it this particular issue was already fixed.

Hi @lostamcimber, I’ll second @odow’s remarks you’ll need to be on the latest versions of JuMP v0.23, Juniper v0.9 and PowerModels v0.19 and then this should work.

I had a quick look at PowerModels and it was not easy to test that MI-SDP would work on some standard problems (e.g. OTS). Conic versions of the on/off constraints are not yet implemented and the extension requires some careful thought around correctness of the formulation.

So keep working at it. In theory this should be working on some models you can propose and let us know if you run into any bugs with Juniper’s support of MI-SDP.

https://github.com/lanl-ansi/PowerModels.jl/issues/815

Thanks for your replies @odow, @ccoffrin. I have got an update regarding the package versions.

I have upgraded my environment to the following:

Environment 1:

Cbc v1.0.0
Ipopt v1.0.2
JuMP v0.23.1
Juniper v0.9.0
PowerModels v0.19.4
SCS v1.1.0

Firstly, I tried to run a ACPPowerModel without a relaxation for my problem and via the solver Ipopt. It threw the following error:

MathOptInterface.AddConstraintNotAllowed{MathOptInterface.ScalarAffineFunction{Float64}, MathOptInterface.GreaterThan{Float64}}: Adding `MathOptInterface.ScalarAffineFunction{Float64}`-in-`MathOptInterface.GreaterThan{Float64}` constraints cannot be performed: MatrixOfConstraints does not allow modifications to be made to the model once
`MOI.Utilities.final_touch` has been called. This is called at the end of
`MOI.copy_to` and in `MOI.Utilities.attach_optimizer` (which is called by
`MOI.optimize!` in a `MOI.Utilities.CachingOptimizer`). In order to be able to
apply modifications to this model, you should add a layer
`MOI.Utilities.CachingOptimizer(MOI.Utilities.Model{Float64}(), model)`
where `model` is the current model. This will automatically empty `model` when
modifications are done after `MOI.Utilities.final_touch` is called and copy the
model again in `MOI.Utilities.attach_optimizer`.
 You may want to use a `CachingOptimizer` in `AUTOMATIC` mode or you may need to call `reset_optimizer` before doing this operation if the `CachingOptimizer` is in `MANUAL` mode.

Stacktrace:
  [1] add_constraint(model::MathOptInterface.Utilities.MatrixOfConstraints{Float64, MathOptInterface.Utilities.MutableSparseMatrixCSC{Float64, Int32, MathOptInterface.Utilities.ZeroBasedIndexing}, MathOptInterface.Utilities.Hyperrectangle{Float64}, Cbc._LPProductOfSets{Float64}}, func::MathOptInterface.ScalarAffineFunction{Float64}, set::MathOptInterface.GreaterThan{Float64})
    @ MathOptInterface.Utilities C:\Users\XXX\.julia\packages\MathOptInterface\FHFUH\src\Utilities\matrix_of_constraints.jl:376
  [2] add_constraint
    @ C:\Users\XXX\.julia\packages\MathOptInterface\FHFUH\src\Utilities\struct_of_constraints.jl:59 [inlined]
  [3] add_constraint
    @ C:\Users\XXX\.julia\packages\MathOptInterface\FHFUH\src\Utilities\model.jl:373 [inlined]
  [4] add_constraint
    @ C:\Users\XXX\.julia\packages\MathOptInterface\FHFUH\src\Utilities\universalfallback.jl:851 [inlined]
  ...

After that, I tried to downgrade Ipopt to v0.9.0 which forces to the following state of the package:

Environment 2:

Cbc v0.9.1
Ipopt v0.9.0
JuMP v0.22.3
Juniper v0.8.0
PowerModels v0.19.4
SCS v0.9.0

These versions resolved the issue with the error above. However, these are the deprecated versions for my initial problem, SDPWRMPowerModel with unrelaxed integer variables.
Furthermore, the second case which I tested was to run a SDPWRMPowerModel with the solver SCS. For that, I have used the first environment as you suggested to use most recent versions. In that case, the following error is thrown:

------------------------------------------------------------------
status:  solved (inaccurate - reached max_iters)
timings: total: 1.24e+002s = setup: 2.84e-003s + solve: 1.24e+002s
	 lin-sys: 3.76e+000s, cones: 1.18e+002s, accel: 4.28e-001s
------------------------------------------------------------------
objective = 7009.723354 (inaccurate)
------------------------------------------------------------------  

┌ Warning: The relaxation is only almost solved.
└ @ Juniper C:\Users\XXX\.julia\packages\Juniper\HEO6p\src\model.jl:110  

MathOptInterface.UnsupportedConstraint{MathOptInterface.VectorAffineFunction{Float64}, MathOptInterface.SecondOrderCone}: `MathOptInterface.VectorAffineFunction{Float64}`-in-`MathOptInterface.SecondOrderCone` constraint is not supported by the model.

Stacktrace:
  [1] bridge_type(b::MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Cbc.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.GenericModel{Float64, MathOptInterface.Utilities.ObjectiveContainer{Float64}, MathOptInterface.Utilities.VariablesContainer{Float64}, Cbc._CbcConstraints{Float64, MathOptInterface.Utilities.MatrixOfConstraints{Float64, MathOptInterface.Utilities.MutableSparseMatrixCSC{Float64, Int32, MathOptInterface.Utilities.ZeroBasedIndexing}, MathOptInterface.Utilities.Hyperrectangle{Float64}, Cbc._LPProductOfSets{Float64}}, MathOptInterface.Utilities.VectorOfConstraints{MathOptInterface.VectorOfVariables, MathOptInterface.SOS1{Float64}}, MathOptInterface.Utilities.VectorOfConstraints{MathOptInterface.VectorOfVariables, MathOptInterface.SOS2{Float64}}}}}}}, F::Type{MathOptInterface.VectorAffineFunction{Float64}}, S::Type{MathOptInterface.SecondOrderCone})
    @ MathOptInterface.Bridges C:\Users\XXX\.julia\packages\MathOptInterface\FHFUH\src\Bridges\lazy_bridge_optimizer.jl:441
  [2] concrete_bridge_type(b::MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Cbc.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.GenericModel{Float64, MathOptInterface.Utilities.ObjectiveContainer{Float64}, MathOptInterface.Utilities.VariablesContainer{Float64}, Cbc._CbcConstraints{Float64, MathOptInterface.Utilities.MatrixOfConstraints{Float64, MathOptInterface.Utilities.MutableSparseMatrixCSC{Float64, Int32, MathOptInterface.Utilities.ZeroBasedIndexing}, MathOptInterface.Utilities.Hyperrectangle{Float64}, Cbc._LPProductOfSets{Float64}}, MathOptInterface.Utilities.VectorOfConstraints{MathOptInterface.VectorOfVariables, MathOptInterface.SOS1{Float64}}, MathOptInterface.Utilities.VectorOfConstraints{MathOptInterface.VectorOfVariables, MathOptInterface.SOS2{Float64}}}}}}}, F::Type{MathOptInterface.VectorAffineFunction{Float64}}, S::Type{MathOptInterface.SecondOrderCone})
    @ MathOptInterface.Bridges.Constraint C:\Users\XXX\.julia\packages\MathOptInterface\FHFUH\src\Bridges\Constraint\bridge.jl:145
  [3] add_constraint(b::MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Cbc.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.GenericModel{Float64, MathOptInterface.Utilities.ObjectiveContainer{Float64}, MathOptInterface.Utilities.VariablesContainer{Float64}, Cbc._CbcConstraints{Float64, MathOptInterface.Utilities.MatrixOfConstraints{Float64, MathOptInterface.Utilities.MutableSparseMatrixCSC{Float64, Int32, MathOptInterface.Utilities.ZeroBasedIndexing}, MathOptInterface.Utilities.Hyperrectangle{Float64}, Cbc._LPProductOfSets{Float64}}, MathOptInterface.Utilities.VectorOfConstraints{MathOptInterface.VectorOfVariables, MathOptInterface.SOS1{Float64}}, MathOptInterface.Utilities.VectorOfConstraints{MathOptInterface.VectorOfVariables, MathOptInterface.SOS2{Float64}}}}}}}, f::MathOptInterface.VectorAffineFunction{Float64}, s::MathOptInterface.SecondOrderCone)
    @ MathOptInterface.Bridges C:\Users\XXX\.julia\packages\MathOptInterface\FHFUH\src\Bridges\bridge_optimizer.jl:1599
...

What can be the reason of the error with Ipopt after the package update above?

@ccoffrin, you mentioned that it should work with the most recent versions. Were you able to run an OTS problem (or any SDP-relaxed OPF problem with unrelaxed binary variables) with SCS through Juniper and the mentioned versions of the rest of the packages?