MathOptInterface and upcoming breaking changes in JuMP 0.19

Specific Domains Optimization (Mathematical)
1

As I recently alluded to, we will be replacing our solver abstraction layer MathProgBase (MPB) with MathOptInterface (MOI). The MOI manual, under construction, lists the key design considerations. At a high level, the main changes are:

  • Constraints are specified in a very generic way (function-in-set). MOI defines standard functions (variablewise identity, affine, quadratic), and sets (nonnegative, cones, {0,1}), and solvers choose which combinations of functions and sets to accept as constraints. This structure makes it trivial to add new classes of constraints to MOI, and can even be done without modifying MOI (and JuMP).

  • MOI has an entirely new status reporting mechanism to explain what caused the solver to stop and how to interpret the output of the solver. MPB’s status symbols were not very informative in error cases (if the solver stopped because of a time limit, is there a feasible solution available?).

  • Solvers can return more than one output vector (“result”).

  • Almost all information (result vectors, statuses, objective bounds, iteration counts) is now accessed in a unified way with Attribute objects. This makes it trivial to ask the solver for any variable-wise or constraint-wise map like an irreducible inconsistent subset. We have attributes for both primal and dual warmstarts now.

  • We refer to all constraints and variables with reference objects, not integer indices. This makes it possible to remove and add variables and constraints if the solver supports doing so.

  • There is now a general way to modify any aspect of the problem which a solver supports changing. Updated coefficients and bounds go through the same modifyconstraint! method.

Overall I’m very happy with the feel of the new interface. It’s the outcome of a lot of intense discussions stemming from the developers’ meetup and beyond. Thanks to everyone who’s participated so far!

As you can imagine, MOI will mean substantial changes for JuMP. JuMP 0.19 will be the first JuMP release to support MOI, and I expect it to be the most breaking release before JuMP 1.0.

What will likely change (in a breaking way) on the JuMP side is:

  • How statuses are reported. We’ll be working with MOI’s much richer status reporting, so you won’t be checking status symbols from the result of solve() anymore.
  • How constraints are represented internally. This will break many JuMP extensions.
  • Duality. MOI aims to define consistent conventions for duals across problem classes. In some cases, the signs may differ from the current MPB conventions.
  • Solutions/result vectors. getvalue may change (for the better) with the introduction of multiple solutions being returned by the solver. I expect to introduce the concept of the value of a variable in a solution to replace the concept of the value of a variable in a model.
  • Solver-independent callbacks will not be supported in JuMP/MOI. Instead, we’ll be making it easier to access the solver’s native callbacks directly.

For all of the breakage, here are the things that will become easy to do following the transition to MOI:

  • Add new constraint types to JuMP (e.g., indicator constraints, complementarity constraints, piecewise linear constraints, bilinear matrix inequalities)
  • Check the feasibility of a solution with respect to the constraints in the model
  • Access the irreducible inconsistent subset (IIS) for infeasible problems when provided by a solver, and similar properties
  • Modify coefficients and right-hand sides in a coherent way
  • Delete constraints (and variables)

These may not be be present in JuMP 0.19, but will become much easier to implement for anyone who needs this functionality.

We plan on continuing to support JuMP/MPB on Julia 0.7 to ease the transition.

The MOI transition is a big undertaking, and we have a lot of opportunities for new and existing contributors to help out, especially on polishing the MOI documentation, writing tests, and writing MOI solver wrappers. If you are interested, join the JuMP-dev channel and we can figure out an appropriate task based on your interests and experience.

Update: As of December 2018, JuMP 0.19-beta is released.

10 Likes
2

I think it would be helpful to make a small document summarizing these changes in the interface as soon as they become clear, and then publish it before releasing the new JuMP, so we can quickly adapt to it by making backup src files and hopefully avoid any breakage period :slight_smile:

2 Likes
3

Definitely. When JuMP is updated for MOI, we’ll have documentation on the breaking changes and how to adapt to them. We will also have a period for testing and feedback before the official 0.18 release.

1 Like
4

Will JuMP depend on both MOI and MPB to handle nonlinear problems for 0.19?

1 Like
5

Yes

6

What is:

7

Have a look at

8

Now that the 0.7 release is rapidly approaching, any idea when you plan to tag the JuMP 0.19 release?

9

I expect we’ll call for public testing of JuMP/MOI in April, and then the release will follow some time after that. Most of the infrastructure work is finished, but we need to polish up and document the JuMP interface.

At this point we plan on supporting JuMP/MPB in Julia 0.7. This will make the transition easier for users so you can first upgrade to Julia 0.7 and then worry about switching from JuMP/MPB to JuMP/MOI. Clearing out JuMP/MPB deprecation warnings on 0.7 is not the top priority at the moment, however. We’d be very happy to get some PRs for that on the release-0.18 branch of JuMP and on the solver packages.

5 Likes