JuMP 0.21.1: CPLEX Error 1811: Attempt to invoke unsupported operation

mike_k · March 27, 2020, 4:48pm

Dear folks,

I try to add LazyConstaintCuts in a MIP model using MOI v0.9.12, CPLEX 12.9 (and it seems generic callbacks are used by default). After several cuts are added the solving procedure stops with the message in the title. Unfortunately, the problem is too big to boil it down to a minimal example. A similar question was asked here but is still unanswered.

Do you have any ideas how to fix that or how to unravel what exactly the problem is?
Is it possible to try it with non-generic callbacks, and if yes how?

Thanks in advance, mike

odow · March 27, 2020, 5:37pm

Please try the fix (Update MOI_callbacks.jl by odow · Pull Request #286 · jump-dev/CPLEX.jl · GitHub) via:

] add CPLEX#odow-patch-1

mike_k · March 28, 2020, 9:48am

Thanks @odow it works, however, it is still strange. Compared to JuMP 0.18.5 it is super slow meaning that in my problem the function which adds cuts ( MOI.submit(model, MOI.LazyConstraint(cb_data), con)) is called significant more often with JuMP 0.21. And also the runtime is significant larger with JuMP 0.21. Here are some outputs for one and the same problem instance:

JuMP 0.21 | CPLEX 12.9 (or 12.8,… it is the same for both version)

*     0+    0                          135.8195      316.6316            133.13%
      0     0      289.0692    10      135.8195 UserPurge2: 1       127  112.83%
      0     0      283.7922    10      135.8195 UserPurge1: 75      213  108.95%
      0     0      282.6926    13      135.8195 UserPurge1: 58      241  108.14%
*     0+    0                          198.3479      282.6926             42.52%
      0     0      279.8942    14      198.3479 UserPurge1: 109     287   41.11%
      0     0      279.7246    12      198.3479 UserPurge1: 6       301   41.03%
      0     0      279.3128    14      198.3479 UserPurge1: 40      320   40.82%
      0     0      279.0657    14      198.3479 UserPurge1: 25      338   40.70%
      0     0      278.7915    15      198.3479 UserPurge1: 85      359   40.56%
      0     0      278.7157    14      198.3479 UserPurge1: 72      370   40.52%
      0     0      278.6908    14      198.3479 UserPurge1: 10      373   40.51%
      0     0      278.6722    15      198.3479 UserPurge1: 64      378   40.50%

runtime: ~150s
# lazy cut calls: 11313
# user cut calls: 30521
# BB explored nodes: 51

JuMP 0.18.5 | CPLEX 12.8

*     0+    0                            0.0000      539.4865              --- 
      0     0      316.6316     6        0.0000      384.4218      118     --- 
*     0+    0                          135.6275      316.6316           133.46%
      0     0      289.8484    10      135.6275      Fract: 1      128  113.71%
      0     0      283.9790    11      135.6275      Fract: 1      141  109.38%
*     0+    0                          172.8079      283.9790            64.33%
*     0+    0                          200.7656      283.9790            41.45%
      0     2      219.9258     5      200.7656      219.9172      297    9.54%

runtime: ~24s
# lazy cuts calls: 1182
# user cuts calls: 3141
# BB explored nodes: 47

It seems that the add-cut functions are called but the cuts are not added. Is it possibly a bug, or just a default setting issue?

odow · April 7, 2020, 3:11pm

JuMP 0.18 used the old version of CPLEX’s callbacks. JuMP 0.20 uses their new generic ones.

Hard to offer more advice without code; you may want to skip adding a lazy constraint at some calls.

mike_k · April 7, 2020, 3:53pm

Yes, I know it is strange. I also have no clue why this happens. I just changed all MOI and JuMP related calls, and the rest of the code remains exactly the same. As I said, unfortunately the project is too large to boil it down to a minimal example. The lazy cuts are Benders optimality cuts and therefore I don’t want to skip them.

odow · April 7, 2020, 6:49pm

It’s probably worth checking if you are getting called at the same point repeatedly, and therefore you’re adding the same cut multiple times.

Wesley_Coelho · August 18, 2020, 4:41pm

Hello,

I’m experiencing the same problem and I didn’t get how to fix it.
I’m using MOI v0.9.12, JuMP v0.21.1, CPLEX v0.6.3 in julia and ILO CPLEX 12.8.

Here’s my code

function my_Lazy_callback_function(cb_data)
   TOL = 1e-6                                                                                        
   for s in S,t in S, u in U, m in N,  f in F, g in F           
      if my_condition == true && callback_value(cb_data, x[s,f,m,u])*callback_value(cb_data, x[t,g,m,u])>0 + TOL 
         con = @build_constraint(x[s,f,m,u] + x[t,g,m,u]  <= 1)
         MOI.submit(my_model, MOI.LazyConstraint(cb_data), con)                                                                                                                                                                            
      end                                                                                
   end
end
set_optimizer_attributes(my_model, "CPX_PARAM_REDUCE" => 0)
set_optimizer_attributes(my_model, "CPX_PARAM_PRELINEAR" => 0)     
MOI.set(my_model, MOI.LazyConstraintCallback(), my_Lazy_callback_function)

And here’s the error I got (after starting the branch and bound process):

CPLEX.CplexError(1811, "CPLEX Error  1811: Attempt to invoke unsupported operation..\n")

Stacktrace:
 [1] return_status_or_throw at /home/wesley/.julia/packages/CPLEX/VVr3o/src/cpx_generic_callbacks.jl:24 [inlined]
 [2] cbrejectcandidate(::CPLEX.CallbackContext, ::Int32, ::Int32, ::Array{Float64,1}, ::Array{Int8,1}, ::Array{Int32,1}, ::Array{Int32,1}, ::Array{Float64,1}) at /home/wesley/.julia/packages/CPLEX/VVr3o/src/cpx_generic_callbacks.jl:161
 [3] submit(::CPLEX.Optimizer, ::MathOptInterface.LazyConstraint{CPLEX.CallbackContext}, ::MathOptInterface.ScalarAffineFunction{Float64}, ::MathOptInterface.LessThan{Float64}) at /home/wesley/.julia/packages/CPLEX/VVr3o/src/MOI/MOI_callbacks.jl:130
 [4] submit(::MathOptInterface.Bridges.LazyBridgeOptimizer{CPLEX.Optimizer}, ::MathOptInterface.LazyConstraint{CPLEX.CallbackContext}, ::MathOptInterface.ScalarAffineFunction{Float64}, ::MathOptInterface.LessThan{Float64}) at /home/wesley/.julia/packages/MathOptInterface/XiH8D/src/Bridges/bridge_optimizer.jl:1195
 [5] submit(::MathOptInterface.Utilities.CachingOptimizer{MathOptInterface.AbstractOptimizer,MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}, ::MathOptInterface.LazyConstraint{CPLEX.CallbackContext}, ::MathOptInterface.ScalarAffineFunction{Float64}, ::MathOptInterface.LessThan{Float64}) at /home/wesley/.julia/packages/MathOptInterface/XiH8D/src/Utilities/cachingoptimizer.jl:667
 [6] submit(::Model, ::MathOptInterface.LazyConstraint{CPLEX.CallbackContext}, ::ScalarConstraint{GenericAffExpr{Float64,VariableRef},MathOptInterface.LessThan{Float64}}) at /home/wesley/.julia/packages/JuMP/CZ8vV/src/callbacks.jl:35
 [7] (::var"#my_Lazy_callback_function#260"{Instance,Array{VariableRef,4}})(::CPLEX.CallbackContext) at /disk2/wesley/Codes/Lazy/include/NSDP_solver.ipynb:In[+4]:632
 [8] (::CPLEX.var"#45#46"{CPLEX.Optimizer})(::CPLEX.CallbackContext, ::Int64) at /home/wesley/.julia/packages/CPLEX/VVr3o/src/MOI/MOI_callbacks.jl:85
 [9] (::CPLEX.var"#43#44"{CPLEX.Optimizer,CPLEX.var"#45#46"{CPLEX.Optimizer}})(::CPLEX.CallbackContext, ::Int64) at /home/wesley/.julia/packages/CPLEX/VVr3o/src/MOI/MOI_callbacks.jl:28
 [10] callback_wrapper(::Ptr{Nothing}, ::Int64, ::Ptr{Nothing}) at /home/wesley/.julia/packages/CPLEX/VVr3o/src/cpx_generic_callbacks.jl:17
 [11] optimize! at /home/wesley/.julia/packages/CPLEX/VVr3o/src/cpx_solve.jl:7 [inlined]
 [12] optimize!(::CPLEX.Optimizer) at /home/wesley/.julia/packages/CPLEX/VVr3o/src/MOI/MOI_wrapper.jl:1838
 [13] optimize!(::MathOptInterface.Bridges.LazyBridgeOptimizer{CPLEX.Optimizer}) at /home/wesley/.julia/packages/MathOptInterface/XiH8D/src/Bridges/bridge_optimizer.jl:239
 [14] optimize!(::MathOptInterface.Utilities.CachingOptimizer{MathOptInterface.AbstractOptimizer,MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}) at /home/wesley/.julia/packages/MathOptInterface/XiH8D/src/Utilities/cachingoptimizer.jl:189
 [15] optimize!(::Model, ::Nothing; bridge_constraints::Bool, ignore_optimize_hook::Bool, kwargs::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at /home/wesley/.julia/packages/JuMP/CZ8vV/src/optimizer_interface.jl:131
 [16] optimize! at /home/wesley/.julia/packages/JuMP/CZ8vV/src/optimizer_interface.jl:107 [inlined] (repeats 2 times)
 [17] macro expansion at /disk2/wesley/Codes/Lazy/include/NSDP_solver.ipynb:In[+10]:9 [inlined]
 [18] macro expansion at ./util.jl:234 [inlined]
 [19] solve_NSDP_model(::Model, ::Parameters, ::Instance) at /disk2/wesley/Codes/Lazy/include/NSDP_solver.ipynb:In[+10]:8
 [20] top-level scope at ./In[2]:126

If a change the LazyConstraint routine by UserCut as described in Callbacks · JuMP, it works. However, I have to use LazyConstraints cause I want to cut some integer solutions (since I initiate the process with a reduced model).

Also, some cuts are added during the pre-processing, but it stops with the above error in the first node of the B&B tree.

Could you please help me?

Regards,
Wesley

odow · August 18, 2020, 8:15pm

Upgrade to CPLEX 12.10. I think CPLEX 12.8 had an issue with the lazy callbacks.

Wesley_Coelho · August 19, 2020, 9:31am

Hello Odow,

Thank you for your advice.
Indeed, the problem seems to come from Cplex 12.8. My code works perfectly with Cplex 12.9.