Callback: solver independent or dependent

Specific Domains Optimization (Mathematical)
1

Hi,

I’m currently diving into my first experience with callbacks and have a few doubts that I hope you can help clarify.
The goal is to implement callbacks that add user cuts to the root node (or subsequent nodes) based on the continuous relaxation values of some variables.

Is the solver-independent callback able to control where the user cuts are added in the branch, or is it necessary to use a solver-dependent callback?

I have followed the example in https://github.com/jump-dev/Gurobi.jl (section Callback), which considers Gurobi’s solver-specific callbacks. Instead of lazy constraints, I want to consider User cut.
Therefore, I adapt the code as follows:

  • I removed cb_where != GRB_CB_MIPSOL because otherwise it returned an error ; in fact, the Gurobi manual says “This routine (user cut) can only be called when the where value on the callback routine is GRB_CB_MIPNODE”. In this way the cut should be add only if the solver is currently exploring a MIP node, and not for example in the presolve.
cb_calls = Cint[]
function my_callback_function(cb_data, cb_where::Cint)
    push!(cb_calls, cb_where)
    if cb_where != GRB_CB_MIPNODE
        return
    end
  • I also added the following condition MIPNODE_NODCNT!=0 to consider the cut only in the root, but it does not seem to work because the parameter MIPNODE_NODCNT only takes the value 5005 (referred to as the callback constant) and thus it never adds the cut
if MIPNODE_NODCNT!=0
        return
    end
  • I considered MOI.UserCut instead of MOI.LazyConstraint
  • Is _val the value that the variable takes in the relaxation?
Gurobi.load_callback_variable_primal(cb_data, cb_where)
variable_val = callback_value(cb_data, variable)
if Condition-val-dependent
   return
end
con = @build_constraint(constraint)
MOI.submit(model, MOI.UserCut(cb_data), con)
end # end of the function

MOI.set(model, Gurobi.CallbackFunction(), my_callback_function)
  • Without the MIPNODE_NODCNT condition, after some iteration and after adding some cuts, I got the error LoadError: Gurobi Error 10005, described as “Attempted to query or set an attribute that could not be accessed at that time”. The error is followed by:
    [1] _check_ret
    [2] load_callback_variable_primal(cb_data::Gurobi.CallbackData, cb_where::Int32)
    Any suggestions?

Thank you in advance,
Martina

2

Hi @martina.gherardi,

Here are some answers to your questions.

No.

if cb_where != GRB_CB_MIPNODE

This looks good

MIPNODE_NODCNT only takes the value 5005 (referred to as the callback constant)

This constant is a callback code https://www.gurobi.com/documentation/current/refman/cb_codes.html
You need to query the associated value with GRBcbget:

    resultP = Ref{Cint}()
    GRBcbget(cb_data, cb_where, MIPNODE_NODCNT, resultP)
    if resultP[] != 0
        return
    end

I considered MOI.UserCut instead of MOI.LazyConstraint

This should be okay

Attempted to query or set an attribute that could not be accessed at that time

As shown in the README example, you need to check the status of the node before querying the primal:

        resultP = Ref{Cint}()
        GRBcbget(cb_data, cb_where, GRB_CB_MIPNODE_STATUS, resultP)
        if resultP[] != GRB_OPTIMAL
            return  # Solution is something other than optimal.
        end
3

Hi @odow, many thanks for your quick response.

Why is it necessary to exit the function when resultP[] != GRB_OPTIMAL?

Are there general examples of more complicated callbacks?
For the _val I would like to consider user cuts based on a fractional solution. Do you know how to do this?

4

Because the solver might not have a solution to return via load_callback_variable_primal. This is why your original code threw the 10005 error.

I would like to consider user cuts based on a fractional solution. Do you know how to do this?

The solution associated with callback_value is a factional solution if the node is MIPNNODE.

Are there general examples of more complicated callbacks?

Not really. But it seems like you’re almost there.

5

Sorry to bother you again.

I added a control on the GRB_CB_MIP_NODCNT which must =0; otherwise return:

resultP = Ref{Cint}()
GRBcbget(cb_data, cb_where, GRB_CB_MIP_NODCNT, resultP)
 if resultP[] != 0     
 return 
 end

Since I do not need values, and therefore I do not need to call load_callback_variable_primal, I can skip both the cb_where != GRB_CB_MIPNODE and the != GRB_OPTIMAL checks.

BUT I was expecting to be able at this point to add a user cut to the root node only (before the root relaxation was solved) but I got the following error:

ERROR: LoadError: Gurobi Error 10011: User cuts only allowed from MIPNODE callback

So the thing is that I cannot add a user cut in the root node only(?)

Thanks

6

I assume so. I haven’t tried to write a callback like this, so I don’t have any suggestions or insight other than what the error says.

7

Hi @odow,
The Gurobi support suggested me that usercuts are only allowed from MIPNODE callback. They also say that the node count is given by GRB_CB_MIPNODE_NODCNT, that should take the value 0 when in the root node.
Therefore I considered the following as a test:

cb_calls = Cint[]
function MyUsercutRelaxedRawSD(cb_data, cb_where::Cint)
      push!(cb_calls, cb_where)
      if cb_where != GRB_CB_MIPNODE
            return
      end
     resultP = Ref{Cint}()
     GRBcbget(cb_data, cb_where, GRB_CB_MIPNODE_NODCNT, resultP)
     println("resultP= ",resultP[])
end

I obtained resultP=0 even if the code is exploring a different node from the root:

Gurobi Optimizer version 11.0.0 build v11.0.0rc2 (win64 - Windows 10.0 (19045.2))

CPU model: AMD Ryzen 9 7950X 16-Core Processor, instruction set [SSE2|AVX|AVX2|AVX512]
Thread count: 8 physical cores, 16 logical processors, using up to 16 threads

Optimize a model with 1809 rows, 1621 columns and 4028 nonzeros
Model fingerprint: 0xc2ebdbd9
Model has 7 quadratic constraints
Model has 360 SOS constraints
Variable types: 1207 continuous, 414 integer (216 binary)
Coefficient statistics:
  Matrix range     [1e-01, 6e+03]
  QMatrix range    [1e-01, 6e+02]
  QLMatrix range   [1e+00, 4e+04]
  Objective range  [1e+00, 1e+00]
  Bounds range     [0e+00, 0e+00]
  RHS range        [1e+00, 2e+04]
Presolve removed 788 rows and 432 columns
Presolve time: 0.02s
Presolved: 1648 rows, 1499 columns, 4896 nonzeros
Presolved model has 188 SOS constraint(s)
Presolved model has 310 bilinear constraint(s)

Solving non-convex MIQCP

Variable types: 979 continuous, 520 integer (345 binary)

Root relaxation: objective 1.203848e+06, 1765 iterations, 0.02 seconds (0.04 work units)

    Nodes    |    Current Node    |     Objective Bounds      |     Work
 Expl Unexpl |  Obj  Depth IntInf | Incumbent    BestBd   Gap | It/Node Time

     0     0 1203848.04    0  327          - 1203848.04      -     -    0s
     0     0 1207046.29    0  327          - 1207046.29      -     -    0s
...
resultP= 0.0
     0     0 1620770.11    0  521          - 1620770.11      -     -    0s
resultP= 0.0
H    0     0                    2026849.1852 1620770.11  20.0%     -    0s
resultP= 0.0
     0     2 1620770.11    0  520 2026849.19 1620770.11  20.0%     -    0s
resultP= 0.0
resultP= 0.0
resultP= 0.0
resultP= 0.0
resultP= 0.0
resultP= 0.0
resultP= 0.0
resultP= 0.0
resultP= 0.0
resultP= 0.0
...
resultP= 0.0
resultP= 0.0
resultP= 0.0
resultP= 0.0
resultP= 0.0
resultP= 0.0
H   36    40                    1971453.8940 1650777.26  16.3%   115    0s
resultP= 0.0
resultP= 0.0
resultP= 0.0
resultP= 0.0
resultP= 0.0
resultP= 0.0

Am i getting the reference or the print wrong?

I also noticed on the gurobi manual that GRB_CB_MIPNODE_NODCNT is declared as double in the result type. Therefore, I modified the code as follows but without any improvement (the result is still 0 even if I am not in the root node):

        resultP = Ref{Cfloat}()
        GRBcbget(cb_data, cb_where, GRB_CB_MIPNODE_NODCNT, resultP)
        println("resultP= ",resultP[])

I obtained another unespected result when asking for GRB_CB_MIPNODE_OBJBND, i.e. the current best objective bound:

        resultPBB = Ref{Cfloat}()
        GRBcbget(cb_data, cb_where, GRB_CB_MIPNODE_OBJBND, resultPBB)
         println("resultPBB = ",resultPBB[])

Why does the result differ from the BestBd shown in the log?

    Nodes    |    Current Node    |     Objective Bounds      |     Work
 Expl Unexpl |  Obj  Depth IntInf | Incumbent    BestBd   Gap | It/Node Time

     0     0 1203848.04    0  327          - 1203848.04      -     -    0s
     0     0 1207046.29    0  327          - 1207046.29      -     -    0s
resultPBB = 799854.4
     0     0 1434920.11    0  379          - 1434920.11      -     -    0s
resultPBB = 716.1741
     0     0 1570002.10    0  235          - 1570002.10      -     -    0s
resultPBB = 3.436603e-19
     0     0 1573422.13    0  238          - 1573422.13      -     -    0s
resultPBB = 3.436603e-19
     0     0 1610590.08    0  377          - 1610590.08      -     -    0s
resultPBB = -1.395841e17
     0     0 1611325.78    0  387          - 1611325.78      -     -    0s
resultPBB = -262242.72
     0     0 1615410.16    0  387          - 1615410.16      -     -    0s
resultPBB = -1.0059933e33
     0     0 1617104.96    0  400          - 1617104.96      -     -    0s
resultPBB = -1.0059933e33
     0     0 1617503.68    0  383          - 1617503.68      -     -    0s
resultPBB = 1.6004671e-21
     0     0 1620770.11    0  385          - 1620770.11      -     -    0s
resultPBB = 1.6004671e-21
     0     0 1620770.11    0  350          - 1620770.11      -     -    0s
resultPBB = 1.6004671e-21
     0     0 1620770.11    0  385          - 1620770.11      -     -    0s
resultPBB = 1.6004671e-21
     0     0 1620770.11    0  577          - 1620770.11      -     -    0s
resultPBB = 1.6004671e-21
     0     0 1620770.11    0  521          - 1620770.11      -     -    0s
resultPBB = 1.6004671e-21
H    0     0                    2026849.1852 1620770.11  20.0%     -    0s
resultPBB = 1.6004671e-21
     0     2 1620770.11    0  520 2026849.19 1620770.11  20.0%     -    0s

Any suggestions would be greatly appreciated, thank you.

1 Like
8

Update: I found the solution.
You need to declare resultP = Ref{Cdouble}()

thank you in any case

1 Like
9

Yes, you need to be a bit careful when working with the C API. Cfloat is a 32-bit number. You did need Cdouble, which is a 64-bit number:

julia> Cfloat
Float32

julia> Cdouble
Float64
1 Like