JuMP/CPLEX: confusion with number of cuts added in callback

Specific Domains Optimization (Mathematical)
1

Dear folks,
I use JuMP (v0.21.4) and CPLEX (v0.7.3) to solve a MILP with lazy and user cuts added in a generic callback. However, I am confused with the output. To obtain the number of cuts after solving, I wrote the function:

function cpx_getnumcuts( model::Model, cuttype::Int )
    moi_model = backend(model)

    data_p = Ref{Cint}()
    ret = CPXgetnumcuts(moi_model.env, moi_model.lp, cuttype, data_p)

    if ret != 0
        @warn "error retrieving $cuttype"
    end
    return data_p[]::Int32
end

cuttype with values CPLEX.CPX_CUT_TABLE or CPLEX.CPX_CUT_USER should return the number of lazy and user cuts, respectively, as far as I understand the cplex documentation. Now there are two issues:
(i) If I deactivate adding user cuts, the function with cuttype = CPLEX.CPX_CUT_TABLE always returns zero whereas the returned value coincides with the CPLEX output “User cuts applied:” if cuttype = CPLEX.CPX_CUT_USER . It is the same behavior when the user cuts are activated.

(ii) When I submit a cut, I also count the number of submissions manually, e.g.,

MOI.submit(model, MOI.LazyConstraint(cb_data), con)
nLCUTS += 1

and

MOI.submit(model, MOI.UserCut(cb_data), con)
nUCUTS += 1

However, the number of cuts never coincide, for instance, here is an output of an instance

nCUTS: 76   #return value of the above function with cuttype = CPLEX.CPX_CUT_USER
nLCUTS: 550
nUCUTS: 17

Which values are correct? And/or how can I retrieve the correct numbers of added lazy and user cuts (separately)?

Many thanks in advance, mike_k

2

The operative word in the documentation is “in use”:

the number of cuts of the specified type in use at the end of the previous optimization

CPLEX can decide not to add cuts that you submit, provided the final solution respects all cuts that were submitted.

I don’t understand why nCUTS is less than nUCUTS though.

3

Thank you odow. However, I still do not understand why the number of lazy constraints is retrieved with CPLEX.CPX_CUT_USER and not with CPLEX.CPX_CUT_TABLE as stated in the documentation. Here is an MWE (the example from the CPLEX.jl page together with the above function):

using JuMP, CPLEX

function cpx_getnumcuts( model::Model, cuttype::Int )
    moi_model = backend( model )

    data_p = Ref{Cint}()
    ret = CPXgetnumcuts(moi_model.env, moi_model.lp, cuttype, data_p)

    if ret != 0
        @warn "error retrieving $cuttype"
    end
    return data_p[]::Int32
end

model = direct_model(CPLEX.Optimizer())
set_silent(model)

# This is very, very important!!! Only use callbacks in single-threaded mode.
MOI.set(model, MOI.NumberOfThreads(), 1)

@variable(model, 0 <= x <= 2.5, Int)
@variable(model, 0 <= y <= 2.5, Int)
@objective(model, Max, y)
cb_calls = Clong[]
function my_callback_function(cb_data::CPLEX.CallbackContext, context_id::Clong)
    # You can reference variables outside the function as normal
    push!(cb_calls, context_id)
    # You can select where the callback is run
    if context_id != CPX_CALLBACKCONTEXT_CANDIDATE
        return
    end
    ispoint_p = Ref{Cint}()
    ret = CPXcallbackcandidateispoint(cb_data, ispoint_p)
    if ret != 0 || ispoint_p[] == 0
        return  # No candidate point available or error
    end
    # You can query CALLBACKINFO items
    valueP = Ref{Cdouble}()
    ret = CPXcallbackgetinfodbl(cb_data, CPXCALLBACKINFO_BEST_BND, valueP)
    @info "Best bound is currently: $(valueP[])"
    # As well as any other C API
    x_p = Vector{Cdouble}(undef, 2)
    obj_p = Ref{Cdouble}()
    ret = CPXcallbackgetincumbent(cb_data, x_p, 0, 1, obj_p)
    if ret == 0
        @info "Objective incumbent is: $(obj_p[])"
        @info "Incumbent solution is: $(x_p)"
        # Use CPLEX.column to map between variable references and the 1-based
        # column.
        x_col = CPLEX.column(cb_data, index(x))
        @info "x = $(x_p[x_col])"
    else
        # Unable to query incumbent.
    end

    # Before querying `callback_value`, you must call:
    CPLEX.load_callback_variable_primal(cb_data, context_id)
    x_val = callback_value(cb_data, x)
    y_val = callback_value(cb_data, y)
    # You can submit solver-independent MathOptInterface attributes such as
    # lazy constraints, user-cuts, and heuristic solutions.
    if y_val - x_val > 1 + 1e-6
        con = @build_constraint(y - x <= 1)
        MOI.submit(model, MOI.LazyConstraint(cb_data), con)
    elseif y_val + x_val > 3 + 1e-6
        con = @build_constraint(y + x <= 3)
        MOI.submit(model, MOI.LazyConstraint(cb_data), con)
    end
end
MOI.set(model, CPLEX.CallbackFunction(), my_callback_function)
optimize!(model)

nLCUTS  = cpx_getnumcuts( model, CPLEX.CPX_CUT_TABLE )
@info "nLCUTS: $nLCUTS "
nUCUTS  = cpx_getnumcuts( model, CPLEX.CPX_CUT_USER )
@info "nUCUTS: $nUCUTS "

yields:

[ Info: Best bound is currently: 2.0
[ Info: Objective incumbent is: 0.0
[ Info: Incumbent solution is: [0.0, 0.0]
[ Info: x = 0.0
[ Info: nLCUTS: 0 
[ Info: nUCUTS: 1
4

MOI.LazyConstraint calls IBM Documentation.

It CPLEX is returning the wrong number, that’s a bug in CPLEX or their documentation. You should contact IBM support. (Note that CPLEX.jl is maintained by the community, and is not a product of IBM.)

5

I think the same problem persists in CPLEX 22.1.1. Although a lazy constraint has been submitted, the CPXgetnumcuts routine thinks that a user cut has been used.

Interestingly, both CPXgetnumusercuts and CPXgetnumlazyconstraints returns zero for the callback example for CPLEX.jl.

As suggested by the CPLEX documentation, these two routines count the number of user cuts and lazy constraints in the problem object. Different from the CPXgetnumcut routine which emphsizes the in use cuts, these two routines should be counting user cuts and lazy constraints that exist.

I’ve tested invoking the routines either within the callback (after the lazy constraint been submitted) or after the model been optimized. The routines only return zero.

1 Like
6

I only have an old version of CPLEX install, but I can confirm.

julia> using JuMP, CPLEX

julia> model = direct_model(CPLEX.Optimizer());

julia> MOI.set(model, MOI.NumberOfThreads(), 1)

julia> @variable(model, 0 <= x <= 2.5, Int)
x

julia> @variable(model, 0 <= y <= 2.5, Int)
y

julia> @objective(model, Max, y)
y

julia> MOI.set(model, CPLEX.CallbackFunction(), (cb_data, context_id) -> begin
           if context_id != CPX_CALLBACKCONTEXT_CANDIDATE
               return
           end
           CPLEX.load_callback_variable_primal(cb_data, context_id)
           x_val = callback_value(cb_data, x)
           y_val = callback_value(cb_data, y)
           if y_val - x_val > 1 + 1e-6
               con = @build_constraint(y - x <= 1)
               MOI.submit(model, MOI.LazyConstraint(cb_data), con)
           elseif y_val + x_val > 3 + 1e-6
               con = @build_constraint(y + x <= 3)
               MOI.submit(model, MOI.LazyConstraint(cb_data), con)
           end
       end)

julia> optimize!(model)
Version identifier: 12.10.0.0 | 2019-11-26 | 843d4de
CPXPARAM_Threads                                 1
Found incumbent of value 0.000000 after 0.00 sec. (0.00 ticks)
Warning:  Non-integral bounds for integer variables rounded.
Tried aggregator 1 time.
Reduced MIP has 0 rows, 2 columns, and 0 nonzeros.
Reduced MIP has 0 binaries, 2 generals, 0 SOSs, and 0 indicators.
Presolve time = 0.00 sec. (0.00 ticks)
Tried aggregator 1 time.
Reduced MIP has 0 rows, 2 columns, and 0 nonzeros.
Reduced MIP has 0 binaries, 2 generals, 0 SOSs, and 0 indicators.
Presolve time = 0.00 sec. (0.00 ticks)
MIP emphasis: balance optimality and feasibility.
MIP search method: dynamic search.
Parallel mode: none, using 1 thread.
Root relaxation solution time = 0.00 sec. (0.00 ticks)

        Nodes                                         Cuts/
   Node  Left     Objective  IInf  Best Integer    Best Bound    ItCnt     Gap

*     0+    0                            0.0000        2.0000              --- 
*     0     0      integral     0        2.0000        2.0000        0    0.00%
Elapsed time = 0.00 sec. (0.00 ticks, tree = 0.00 MB, solutions = 2)

User cuts applied:  1

Root node processing (before b&c):
  Real time             =    0.00 sec. (0.00 ticks)
Sequential b&c:
  Real time             =    0.00 sec. (0.00 ticks)
                          ------------
Total (root+branch&cut) =    0.00 sec. (0.00 ticks)

julia> cpx = backend(model)
Ptr{Nothing} @0x00007feaaf57eaf0

julia> CPXgetnumlazyconstraints(cpx.env, cpx.lp)
0

julia> CPXgetnumusercuts(cpx.env, cpx.lp)
0

This seems like a bug in CPLEX. (The log says User cuts applied: 1.) But you’d need to contact IBM support to confirm.

Ultimately, you’re probably better off just recording how many cuts you add yourself in Julia.

7

Hi Oscar,

Thanks for your time!

I agree that this is most likely a bug in CPLEX, at least for the C API.

1 Like