# \[ANN\] Upcoming breaking changes to CPLEX.jl and Gurobi.jl

**URL:** <https://discourse.julialang.org/t/ann-upcoming-breaking-changes-to-cplex-jl-and-gurobi-jl/47814>\
**Category:** Optimization (Mathematical)\
**Tags:** jump\
**Created:** [October 5, 2020, 9:56pm UTC](https://discourse.julialang.org/t/ann-upcoming-breaking-changes-to-cplex-jl-and-gurobi-jl/47814 "2020-10-05T21:56:14Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [October 5, 2020, 9:56pm UTC](https://discourse.julialang.org/t/ann-upcoming-breaking-changes-to-cplex-jl-and-gurobi-jl/47814/1 "2020-10-05T21:56:14Z")

</div>

**Update: 20 October 2020: New versions of CPLEX.jl and Gurobi.jl are tagged.**

_tl;dr: CPLEX.jl and Gurobi.jl ~~will~~ have major breaking changes in the ~~next~~ latest release. If you just use `CPLEX.Optimizer()` or `Gurobi.Optimizer()` with JuMP, nothing changes. If you use **any** lower-level functionality in CPLEX.jl or Gurobi.jl (e.g., solver-dependent callbacks, or functions from the C API), your code will break. Furthermore, all support for MathProgBase has been removed. ~~You can try out the changes by adding the `master` branch of each package. We will wait at least two weeks before tagging to give time for feedback. If no issues are raised during that time, we will tag new releases on October 20.~~ _

Dear all,

We have some big breaking changes landing in the next releases of [CPLEX.jl](https://github.com/jump-dev/CPLEX.jl) and [Gurobi.jl](https://github.com/jump-dev/Gurobi.jl) wrappers.

**All low-level functions in the wrapper have been removed and replaced by a complete rewrite using Clang.jl. If you used any of these functions, your code will break. In addition, all support for MathProgBase has been removed.**

## Why did we do this?

CPLEX and Gurobi are two of the oldest solver wrappers, and the C wrappers have evolved on an ad-hoc basis over time. (The first [commit with working code](https://github.com/jump-dev/Gurobi.jl/commit/300725f809eff25c6564ac3bc42c7bcf3df2b35a) was made to Gurobi.jl on March 9, 2013!) This meant that we had frequent issues with “missing” functions in the C API that were not wrapped.

The age of the code also meant that the naming scheme was inconsistent. A C function like `GRBaddqpterms` might be called `add_qpterms!`, `add_qp_terms`, or `c_api_addqpterms`. In addition, the arguments to the functions were often different to the underlying C API and some arguments were not exposed. Combined with the lack of documentation, this meant it was hard to identify which Julia function matched which C function, and which arguments to supply.

The main reason you would want to use the C API is in a solver-dependent callback. In addition to now supporting the full functionality of a solver-dependent callback, we now have documentation. See the solver-dependent callbacks section below.

## What did we do?

We used [Clang.jl](https://github.com/JuliaInterop/Clang.jl) to wrap the complete C API of each solver.

The functions now have a consistent naming scheme, the arguments are all available and consistent, and the complete API is already documented by each solver.

In addition, _all_ support for [MathProgBase](https://github.com/JuliaOpt/MathProgBase.jl) has been removed.

## How do I try it out?

To try out the latest changes, use the following:

```julia
import Pkg
Pkg.add(Pkg.PackageSpec(name = "CPLEX", rev = "master"))
# or
Pkg.add(Pkg.PackageSpec(name = "Gurobi", rev = "master"))

```

Then restart Julia for the change to take effect.

If your code breaks, and you want to revert to the old API, use:

```julia
import Pkg
Pkg.add(Pkg.PackageSpec(name = "CPLEX", version = v"0.6"))
# or
Pkg.add(Pkg.PackageSpec(name = "Gurobi", version = v"0.8"))

```

Then restart Julia for the change to take effect.

If you have comments or feedback, please open an issue:

- [Sign in to GitHub · GitHub](https://github.com/jump-dev/CPLEX.jl/issues/new)
- [Sign in to GitHub · GitHub](https://github.com/jump-dev/Gurobi.jl/issues/new)

## When will new versions be tagged?

We will wait at least two weeks before tagging new versions to give people time to try out the new versions and provide feedback. If no substantive issues are raised, we will tag new releases on October 20.

## CPLEX Changes

**New minimum version requirement:** CPLEX.jl now requires CPLEX version 12.10.

CPLEX has [extensive documentation for their C API](https://www.ibm.com/support/knowledgecenter/SSSA5P_12.10.0/ilog.odms.cplex.help/refcallablelibrary/groups/homepagecallable.html).

The main changes are:

- Function names have changed. For example, `CPLEX.close_CPLEX(env)` is now `CPXcloseCPLEX(env)`.
- For users of `CPLEX.Optimizer`, `model.inner` has been replaced by the fields `model.env` and `model.lp`, which correspond to the environment and problem pointers at the C API level. For example:

```julia
model = direct_model(CPLEX.Optimizer())
cpx_model = backend(model)
stat = CPLEX.get_status(cpx_model.inner)

```

is now:

```Julia
model = direct_model(CPLEX.Optimizer())
cpx_model = backend(model)
stat = CPXgetstat(cpx_model.env, cpx_model.lp)

```

- Querying functionality has changed. For example:

```julia
is_point = CPLEX.cbcandidateispoint(cb_data)

```

is now:

```julia
is_point_P = Ref{Cint}()
ret = CPXcallbackcandidateispoint(cb_data, is_point_P)
if ret != 0
    # Do something because the call failed
end
is_point = is_point_P[]

```

## Gurobi changes

**New minimum version requirement:** Gurobi.jl now requires Gurobi version 9.0 or higher.

Gurobi has [extensive documentation for their C API](https://www.gurobi.com/documentation/9.0/refman/c_api_details.html).

The main changes are:

- Constants have changed. For example `CB_MIPNODE` is now `GRB_CB_MIPNODE` to match the C API.
- Function names have changed. For example `free_env(env)` is now `GRBfreeenv(env)`.
- For users of `Gurobi.Optimizer()`, `model.inner` is now a pointer to the C model, instead of a `Gurobi.Model` object. However, conversion means that you should always pass `model` instead of `model.inner` to the low-level functions. For example:

```julia
model = direct_model(Gurobi.Optimizer())
grb_model = backend(model) # grb_model is Gurobi.Optimizer
# Old
Gurobi.tune_model(grb_model.inner)
# New
GRBtunemodel(grb_model)

```

- Some functions have been removed entirely. For example:

```julia
using JuMP, Gurobi
model = direct_model(Gurobi.Optimizer())
optimize!(model)
grb_model = backend(model)
stat = Gurobi.get_status_code(grb_model.inner)

```

is now:

```julia
using JuMP, Gurobi
model = direct_model(Gurobi.Optimizer())
optimize!(model)
valueP = Ref{Cint}()
grb_model = backend(model)
ret = GRBgetintattr(grb_model, "Status", valueP)
if ret != 0
    # Do something because the call failed
end
stat = valueP[]

```

## Solver-dependent callbacks

Here is an example with the latest syntax for CPLEX:

```julia
using JuMP, CPLEX, Test

model = direct_model(CPLEX.Optimizer())
set_silent(model)

# This is very, very important!!! Only use callbacks in single-threaded mode.
MOI.set(model, MOI.NumberOfThreads(), 1)

@variable(model, 0 <= x <= 2.5, Int)
@variable(model, 0 <= y <= 2.5, Int)
@objective(model, Max, y)
cb_calls = Clong[]
function my_callback_function(cb_data::CPLEX.CallbackContext, context_id::Clong)
    # You can reference variables outside the function as normal
    push!(cb_calls, context_id)
    # You can select where the callback is run
    if context_id != CPX_CALLBACKCONTEXT_CANDIDATE
        return
    end
    # You can query CALLBACKINFO items
    valueP = Ref{Cdouble}()
    ret = CPXcallbackgetinfodbl(cb_data, CPXCALLBACKINFO_BEST_BND, valueP)
    @info "Best bound is currently: $(valueP[])"
    # As well as any other C API
    x_p = Vector{Cdouble}(undef, 2)
    obj_p = Ref{Cdouble}()
    ret = CPXcallbackgetincumbent(cb_data, x_p, 0, 1, obj_p)
    if ret == 0
        @info "Objective incumbent is: $(obj_p[])"
        @info "Incumbent solution is: $(x_p)"
        # Use CPLEX.column to map between variable references and the 1-based
        # column.
        x_col = CPLEX.column(cb_data, index(x))
        @info "x = $(x_p[x_col])"
    else
        # Unable to query incumbent.
    end

    # Before querying `callback_value`, you must call:
    CPLEX.load_callback_variable_primal(cb_data, context_id)
    x_val = callback_value(cb_data, x)
    y_val = callback_value(cb_data, y)
    # You can submit solver-independent MathOptInterface attributes such as
    # lazy constraints, user-cuts, and heuristic solutions.
    if y_val - x_val > 1 + 1e-6
        con = @build_constraint(y - x <= 1)
        MOI.submit(model, MOI.LazyConstraint(cb_data), con)
    elseif y_val + x_val > 3 + 1e-6
        con = @build_constraint(y + x <= 3)
        MOI.submit(model, MOI.LazyConstraint(cb_data), con)
    end
end
MOI.set(model, CPLEX.CallbackFunction(), my_callback_function)
optimize!(model)
@test termination_status(model) == MOI.OPTIMAL
@test primal_status(model) == MOI.FEASIBLE_POINT
@test value(x) == 1
@test value(y) == 2

```

Here is an example with the latest syntax for Gurobi:

```julia
using JuMP, Gurobi, Test

model = direct_model(Gurobi.Optimizer())
@variable(model, 0 <= x <= 2.5, Int)
@variable(model, 0 <= y <= 2.5, Int)
@objective(model, Max, y)
cb_calls = Cint[]
function my_callback_function(cb_data, cb_where::Cint)
    # You can reference variables outside the function as normal
    push!(cb_calls, cb_where)
    # You can select where the callback is run
    if cb_where != GRB_CB_MIPSOL && cb_where != GRB_CB_MIPNODE
        return
    end
    # You can query a callback attribute using GRBcbget
    if cb_where == GRB_CB_MIPNODE
        resultP = Ref{Cint}()
        GRBcbget(cb_data, cb_where, GRB_CB_MIPNODE_STATUS, resultP)
        if resultP[] != GRB_OPTIMAL
            return # Solution is something other than optimal.
        end
    end
    # Before querying `callback_value`, you must call:
    Gurobi.load_callback_variable_primal(cb_data, cb_where)
    x_val = callback_value(cb_data, x)
    y_val = callback_value(cb_data, y)
    # You can submit solver-independent MathOptInterface attributes such as
    # lazy constraints, user-cuts, and heuristic solutions.
    if y_val - x_val > 1 + 1e-6
        con = @build_constraint(y - x <= 1)
        MOI.submit(model, MOI.LazyConstraint(cb_data), con)
    elseif y_val + x_val > 3 + 1e-6
        con = @build_constraint(y + x <= 3)
        MOI.submit(model, MOI.LazyConstraint(cb_data), con)
    end
end
# You _must_ set this parameter if using lazy constraints.
MOI.set(model, MOI.RawParameter("LazyConstraints"), 1)
MOI.set(model, Gurobi.CallbackFunction(), my_callback_function)
optimize!(model)
@test termination_status(model) == MOI.OPTIMAL
@test primal_status(model) == MOI.FEASIBLE_POINT
@test value(x) == 1
@test value(y) == 2

```

## Other changes

The upcoming releases also contain numerous improvements to the MathOptInterface wrappers of both solvers. For example, Gurobi gained [support for indicator constraints](https://github.com/jump-dev/Gurobi.jl/pull/352) and a [solve can now be interrupted via CTRL+C](https://github.com/jump-dev/Gurobi.jl/pull/349). In CPLEX, binary variables without explicit bounds [no longer throw a warning](https://github.com/jump-dev/CPLEX.jl/pull/318), and [dual variables are returned for quadratic constraints](https://github.com/jump-dev/CPLEX.jl/pull/321).

Thanks,  
Oscar

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [October 5, 2020, 9:56pm UTC](https://discourse.julialang.org/t/ann-upcoming-breaking-changes-to-cplex-jl-and-gurobi-jl/47814/2 "2020-10-05T21:56:40Z")

</div>



---

<div class="post-metadata">

**Author:** ![GerhardTotschnig](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gerhardtotschnig/32/215516_2.png) [@GerhardTotschnig](https://discourse.julialang.org/u/GerhardTotschnig)\
**Post date:** [October 15, 2020, 9:37pm UTC](https://discourse.julialang.org/t/ann-upcoming-breaking-changes-to-cplex-jl-and-gurobi-jl/47814/3 "2020-10-15T21:37:46Z")

</div>

For me (Julia beginner, AMPL oldtimer) it is not clear how to use  
the normal cplex routines:

```julia
using JuMP, XLSX, CPLEX

m = Model(CPLEX.Optimizer)

CPXwriteprob( m.env, m.lp, "mymodel", "LP" )
CPXXsetlogfilename( m.env, "mylogfile.txt", "a") 
CPXrefineconflict( m.env, m.lp, NULL, NULL);

```

I get an error:  
`ERROR: LoadError: type Model has no field env`

I use CPLEX 12.10 and CPLEX.jl 0.6.6

Would be great if you can tell me how to call the Cplex routines.  
Best greetings Gerhard

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [October 15, 2020, 9:45pm UTC](https://discourse.julialang.org/t/ann-upcoming-breaking-changes-to-cplex-jl-and-gurobi-jl/47814/4 "2020-10-15T21:45:56Z")

</div>

> For me (Julia beginner, AMPL oldtimer) it is not clear how to use the normal cplex routines:

These routines are for advanced users only. It is expected you have a good understanding of the CPLEX C API.

I have updated the post above to clarify you need to use

```Julia
model = direct_model(CPLEX.Optimizer())
m = backend(model)

```

instead of

```Julia
m = Model(CPLEX.Optimizer)

```

Also note that `NULL` is `C_NULL`, and these function are only available on `CPLEX#master`, not `0.6.6`.

---

<div class="post-metadata">

**Author:** ![GerhardTotschnig](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gerhardtotschnig/32/215516_2.png) [@GerhardTotschnig](https://discourse.julialang.org/u/GerhardTotschnig)\
**Post date:** [October 16, 2020, 8:24am UTC](https://discourse.julialang.org/t/ann-upcoming-breaking-changes-to-cplex-jl-and-gurobi-jl/47814/5 "2020-10-16T08:24:35Z")

</div>

Thank you very much for the help!

It worked as you described with

```julia
model = direct_model(CPLEX.Optimizer())
cpx_model = backend(model)
CPXsetlogfilename(cpx_model.env, "mylogfile.txt", "a")
stat = CPXgetstat(cpx_model.env, cpx_model.lp)

```

but with

```julia
m = Model(CPLEX.Optimizer)

```

it did not work.  
`ERROR: type CachingOptimizer has no field env`

So just for completeness: Is there a way to access CPLEX routines also in CachingOptimizer - Model(CPLEX.Optimizer)?

---

<div class="post-metadata">

**Author:** ![GerhardTotschnig](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gerhardtotschnig/32/215516_2.png) [@GerhardTotschnig](https://discourse.julialang.org/u/GerhardTotschnig)\
**Post date:** [October 16, 2020, 8:26am UTC](https://discourse.julialang.org/t/ann-upcoming-breaking-changes-to-cplex-jl-and-gurobi-jl/47814/6 "2020-10-16T08:26:07Z")

</div>

Thank you very much for the help!  
Please also add this info given above (on how to use CPLEX routines), also to the cplex.jl documentation. The use of solver routines is essential e.g. for trouble shooting an infeasible problem using IIS. Best Greetings Gerhard

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [October 18, 2020, 8:39pm UTC](https://discourse.julialang.org/t/ann-upcoming-breaking-changes-to-cplex-jl-and-gurobi-jl/47814/7 "2020-10-18T20:39:05Z")

</div>

You can use `backend(model).optimizer.model.env` and `backend(model).optimizer.model.lp`, but I don’t recommend this, nor will we document it, since it relies on internal JuMP representations. Use `direct_model` instead.

> The use of solver routines is essential e.g. for trouble shooting an infeasible problem using IIS.

JuMP has documented support for accessing the IIS: [https://jump.dev/JuMP.jl/stable/solutions/#Conflicts-1](https://jump.dev/JuMP.jl/stable/solutions/#Conflicts-1). The main reason to use the C API is for solver-specific callbacks.

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [October 21, 2020, 1:04am UTC](https://discourse.julialang.org/t/ann-upcoming-breaking-changes-to-cplex-jl-and-gurobi-jl/47814/8 "2020-10-21T01:04:47Z")

</div>

Following up on this, new releases for both packages are now tagged and available in the General registry.

The latest versions are:

- CPLEX.jl v0.7.0
- Gurobi.jl v0.9.0

Please open an issue if you experience issues.