Update: 20 October 2020: New versions of CPLEX.jl and Gurobi.jl are tagged.
tl;dr: CPLEX.jl and Gurobi.jl will have major breaking changes in the next latest release. If you just use CPLEX.Optimizer() or Gurobi.Optimizer() with JuMP, nothing changes. If you use any lower-level functionality in CPLEX.jl or Gurobi.jl (e.g., solver-dependent callbacks, or functions from the C API), your code will break. Furthermore, all support for MathProgBase has been removed. You can try out the changes by adding the master branch of each package. We will wait at least two weeks before tagging to give time for feedback. If no issues are raised during that time, we will tag new releases on October 20.
Dear all,
We have some big breaking changes landing in the next releases of CPLEX.jl and Gurobi.jl wrappers.
All low-level functions in the wrapper have been removed and replaced by a complete rewrite using Clang.jl. If you used any of these functions, your code will break. In addition, all support for MathProgBase has been removed.
Why did we do this?
CPLEX and Gurobi are two of the oldest solver wrappers, and the C wrappers have evolved on an ad-hoc basis over time. (The first commit with working code was made to Gurobi.jl on March 9, 2013!) This meant that we had frequent issues with “missing” functions in the C API that were not wrapped.
The age of the code also meant that the naming scheme was inconsistent. A C function like GRBaddqpterms might be called add_qpterms!, add_qp_terms, or c_api_addqpterms. In addition, the arguments to the functions were often different to the underlying C API and some arguments were not exposed. Combined with the lack of documentation, this meant it was hard to identify which Julia function matched which C function, and which arguments to supply.
The main reason you would want to use the C API is in a solver-dependent callback. In addition to now supporting the full functionality of a solver-dependent callback, we now have documentation. See the solver-dependent callbacks section below.
What did we do?
We used Clang.jl to wrap the complete C API of each solver.
The functions now have a consistent naming scheme, the arguments are all available and consistent, and the complete API is already documented by each solver.
In addition, all support for MathProgBase has been removed.
How do I try it out?
To try out the latest changes, use the following:
import Pkg
Pkg.add(Pkg.PackageSpec(name = "CPLEX", rev = "master"))
# or
Pkg.add(Pkg.PackageSpec(name = "Gurobi", rev = "master"))
Then restart Julia for the change to take effect.
If your code breaks, and you want to revert to the old API, use:
import Pkg
Pkg.add(Pkg.PackageSpec(name = "CPLEX", version = v"0.6"))
# or
Pkg.add(Pkg.PackageSpec(name = "Gurobi", version = v"0.8"))
Then restart Julia for the change to take effect.
If you have comments or feedback, please open an issue:
When will new versions be tagged?
We will wait at least two weeks before tagging new versions to give people time to try out the new versions and provide feedback. If no substantive issues are raised, we will tag new releases on October 20.
CPLEX Changes
New minimum version requirement: CPLEX.jl now requires CPLEX version 12.10.
CPLEX has extensive documentation for their C API.
The main changes are:
- Function names have changed. For example,
CPLEX.close_CPLEX(env)is nowCPXcloseCPLEX(env). - For users of
CPLEX.Optimizer,model.innerhas been replaced by the fieldsmodel.envandmodel.lp, which correspond to the environment and problem pointers at the C API level. For example:
is now:model = direct_model(CPLEX.Optimizer()) cpx_model = backend(model) stat = CPLEX.get_status(cpx_model.inner)model = direct_model(CPLEX.Optimizer()) cpx_model = backend(model) stat = CPXgetstat(cpx_model.env, cpx_model.lp) - Querying functionality has changed. For example:
is now:is_point = CPLEX.cbcandidateispoint(cb_data)is_point_P = Ref{Cint}() ret = CPXcallbackcandidateispoint(cb_data, is_point_P) if ret != 0 # Do something because the call failed end is_point = is_point_P[]
Gurobi changes
New minimum version requirement: Gurobi.jl now requires Gurobi version 9.0 or higher.
Gurobi has extensive documentation for their C API.
The main changes are:
- Constants have changed. For example
CB_MIPNODEis nowGRB_CB_MIPNODEto match the C API. - Function names have changed. For example
free_env(env)is nowGRBfreeenv(env). - For users of
Gurobi.Optimizer(),model.inneris now a pointer to the C model, instead of aGurobi.Modelobject. However, conversion means that you should always passmodelinstead ofmodel.innerto the low-level functions. For example:model = direct_model(Gurobi.Optimizer()) grb_model = backend(model) # grb_model is Gurobi.Optimizer # Old Gurobi.tune_model(grb_model.inner) # New GRBtunemodel(grb_model) - Some functions have been removed entirely. For example:
is now:using JuMP, Gurobi model = direct_model(Gurobi.Optimizer()) optimize!(model) grb_model = backend(model) stat = Gurobi.get_status_code(grb_model.inner)using JuMP, Gurobi model = direct_model(Gurobi.Optimizer()) optimize!(model) valueP = Ref{Cint}() grb_model = backend(model) ret = GRBgetintattr(grb_model, "Status", valueP) if ret != 0 # Do something because the call failed end stat = valueP[]
Solver-dependent callbacks
Here is an example with the latest syntax for CPLEX:
using JuMP, CPLEX, Test
model = direct_model(CPLEX.Optimizer())
set_silent(model)
# This is very, very important!!! Only use callbacks in single-threaded mode.
MOI.set(model, MOI.NumberOfThreads(), 1)
@variable(model, 0 <= x <= 2.5, Int)
@variable(model, 0 <= y <= 2.5, Int)
@objective(model, Max, y)
cb_calls = Clong[]
function my_callback_function(cb_data::CPLEX.CallbackContext, context_id::Clong)
# You can reference variables outside the function as normal
push!(cb_calls, context_id)
# You can select where the callback is run
if context_id != CPX_CALLBACKCONTEXT_CANDIDATE
return
end
# You can query CALLBACKINFO items
valueP = Ref{Cdouble}()
ret = CPXcallbackgetinfodbl(cb_data, CPXCALLBACKINFO_BEST_BND, valueP)
@info "Best bound is currently: $(valueP[])"
# As well as any other C API
x_p = Vector{Cdouble}(undef, 2)
obj_p = Ref{Cdouble}()
ret = CPXcallbackgetincumbent(cb_data, x_p, 0, 1, obj_p)
if ret == 0
@info "Objective incumbent is: $(obj_p[])"
@info "Incumbent solution is: $(x_p)"
# Use CPLEX.column to map between variable references and the 1-based
# column.
x_col = CPLEX.column(cb_data, index(x))
@info "x = $(x_p[x_col])"
else
# Unable to query incumbent.
end
# Before querying `callback_value`, you must call:
CPLEX.load_callback_variable_primal(cb_data, context_id)
x_val = callback_value(cb_data, x)
y_val = callback_value(cb_data, y)
# You can submit solver-independent MathOptInterface attributes such as
# lazy constraints, user-cuts, and heuristic solutions.
if y_val - x_val > 1 + 1e-6
con = @build_constraint(y - x <= 1)
MOI.submit(model, MOI.LazyConstraint(cb_data), con)
elseif y_val + x_val > 3 + 1e-6
con = @build_constraint(y + x <= 3)
MOI.submit(model, MOI.LazyConstraint(cb_data), con)
end
end
MOI.set(model, CPLEX.CallbackFunction(), my_callback_function)
optimize!(model)
@test termination_status(model) == MOI.OPTIMAL
@test primal_status(model) == MOI.FEASIBLE_POINT
@test value(x) == 1
@test value(y) == 2
Here is an example with the latest syntax for Gurobi:
using JuMP, Gurobi, Test
model = direct_model(Gurobi.Optimizer())
@variable(model, 0 <= x <= 2.5, Int)
@variable(model, 0 <= y <= 2.5, Int)
@objective(model, Max, y)
cb_calls = Cint[]
function my_callback_function(cb_data, cb_where::Cint)
# You can reference variables outside the function as normal
push!(cb_calls, cb_where)
# You can select where the callback is run
if cb_where != GRB_CB_MIPSOL && cb_where != GRB_CB_MIPNODE
return
end
# You can query a callback attribute using GRBcbget
if cb_where == GRB_CB_MIPNODE
resultP = Ref{Cint}()
GRBcbget(cb_data, cb_where, GRB_CB_MIPNODE_STATUS, resultP)
if resultP[] != GRB_OPTIMAL
return # Solution is something other than optimal.
end
end
# Before querying `callback_value`, you must call:
Gurobi.load_callback_variable_primal(cb_data, cb_where)
x_val = callback_value(cb_data, x)
y_val = callback_value(cb_data, y)
# You can submit solver-independent MathOptInterface attributes such as
# lazy constraints, user-cuts, and heuristic solutions.
if y_val - x_val > 1 + 1e-6
con = @build_constraint(y - x <= 1)
MOI.submit(model, MOI.LazyConstraint(cb_data), con)
elseif y_val + x_val > 3 + 1e-6
con = @build_constraint(y + x <= 3)
MOI.submit(model, MOI.LazyConstraint(cb_data), con)
end
end
# You _must_ set this parameter if using lazy constraints.
MOI.set(model, MOI.RawParameter("LazyConstraints"), 1)
MOI.set(model, Gurobi.CallbackFunction(), my_callback_function)
optimize!(model)
@test termination_status(model) == MOI.OPTIMAL
@test primal_status(model) == MOI.FEASIBLE_POINT
@test value(x) == 1
@test value(y) == 2
Other changes
The upcoming releases also contain numerous improvements to the MathOptInterface wrappers of both solvers. For example, Gurobi gained support for indicator constraints and a solve can now be interrupted via CTRL+C. In CPLEX, binary variables without explicit bounds no longer throw a warning, and dual variables are returned for quadratic constraints.
Thanks,
Oscar