ANN: upcoming refactoring of JuMP's nonlinear API

Specific Domains Optimization (Mathematical)
1

tl;dr:

The next feature release of JuMP (probably v1.2.0) introduces a large
refactoring of JuMP’s nonlinear API. If you have code that accessed private
features of the nonlinear API, such as JuMP._Derivatives or model.nlp_data,
your code will break. If you used only the public, documented API such as
register, @NLconstraint and num_nonlinear_constraints, this change does
not affect you. To try the uncoming release, use
import Pkg; Pkg.pkg"add JuMP#od/moi-nonlinear", then restart Julia for the change to take effect.

The relevant pull request is Refactor to use MOI.Nonlinear by odow · Pull Request #2955 · jump-dev/JuMP.jl · GitHub

What are we doing?

Over the last few months, we have been refactoring how JuMP supports nonlinear
programs. This involved moving and re-organizing a large amount of code from
JuMP into MathOptInterface.

The result is the new MOI.Nonlinear submodule in MathOptInterface, with a
documented public API for creating and dealing with nonlinear programs. Read
more about it here: Overview · MathOptInterface

However, as part of this work we are removing code from JuMP. This code was
internal, undocumented, and not intended for public use. Most of it was
contained in the JuMP._Derivatives submodule, but we also made changes such
as removing model.nlp_data.

Why did we do this?

The nonlinear code in JuMP was a clear example of technical debt. It was
complicated, convoluted, and largely undocumented. People wanting to extend JuMP
for nonlinear programs were forced to use a range of hacks that relied on
undocumented internals.

The new MOI.Nonlinear submodule offers a stable, documented, and public API
for people to build JuMP extensions on. It also enables new features like
swappable automatic differentiation backends, and hessians of user-defined
functions.

We originally considered that any change in the nonlinear API would be a
breaking v2.0.0 release of JuMP and occur at least two years after the release
of JuMP 1.0. However, we implemented the changes quicker than expected, and we were
able to do so in a way that does not break the public nonlinear API. Therefore,
we elected to classify this as a non-breaking feature release.

Does it affect me?

If you have any code that called private features of the JuMP nonlinear API,
such as JuMP._Derivatives or model.nlp_data, your code will break.

If you used only the public, documented API such as register, @NLconstraint
and num_nonlinear_constraints, this does not affect you.

What are the next steps

Try the uncoming release as follows:
import Pkg; Pkg.pkg"add JuMP#od/moi-nonlinear".

If you find any bugs or changes in performance, please post below, or open a
GitHub issue. Once we’re happy that there are no issues with the changes, we
will merge the pull request, and then release a new version of JuMP with the changes.

27 Likes
3

Very cool, thanks for your work on this! :slight_smile:

I’m looking at Overview · MathOptInterface and API Reference · MathOptInterface, it looks like the function is taking multiple arguments x.... Does user-defined functions from arrays to arrays also work in this new interface?

3 Likes
4

Awesome work!

How does that correspond to:

  • Goal: limit the scope to improve robustness Most other AD packages accept arbitrary Julia functions as input and then trace an expression graph using operator overloading. This means they must deal (or detect and ignore) with control flow, I/O, and other vagaries of Julia. In contrast, ReverseAD only accepts functions in the form of Nonlinear.Expression, which greatly limits the range of syntax that it must deal with. By reducing the scope of what we accept as input to functions relevant for mathematical optimization, we can provide a simpler implementation with various performance optimizations.

So you can swap AD backends, but it still needs to be something that only uses registered functions in Nonlinear.Expressions, where this AD backend changes the way the derivative is created for the registered function?

1 Like
5

No. Nothing changes from the current JuMP interface, which requires splatting.

6

Each AD gets given the full expression graph, along with callbacks for how to numerically evaluate the function, gradient, a hessian of each operator. Then they have to produce a MOI.AbstractNLPEvaluator which implements the MOI callbacks like eval_constraint_jacobian and MOI.hessian_lagrangian_structure.

We have a few implementations already which gives us confidence this is a reasonable thing to do:

For @ccoffrin’s AC-OPF problems, SymbolicAD is 3-5x faster than SparseReverseAD, but on other problems it can be much worse.

4 Likes
7

I understand that technical making this a minor release is correct, but on the other hand my sense is that there are packages out there that depend on JuMP and that rely on internals that will break if this gets released as 1.2, right? Complementarity.jl is one that comes to mind, not sure whether there are others as well?

In my mind, in such a situation it would be more helpful for users to just make this a 2.0 release. If I’m a user of a package that depends on JuMP internals, I a) might not even be aware of that, and b) can’t really do anything about it. In the end the user experience will just be that things don’t work, and I think it would be nicer to try to prevent that.

Plus, I’m not aware how a depending package like Complementarity.jl could even signal that it doesn’t work with JuMP 1.2, but it does work with 1.1? Maybe I’m missing something there…

Would there be any real downside of just making this a 2.0?

And also: awesome work, very excited to try it!

4 Likes
8

Packages that depend on internal JuMP APIs should indicate compatibility only with patch releases. I believe the tilde specifier should work. It’s important that we make sure any potentially broken packages have accurate compatibility specs.

1 Like
9

I second Miles’ remarks and would add, given that JuMP has been extremely stable since v0.18 with only minor breaking changes in the last several years leading up to v1.0. Updating to v2.0 with no breaking changes to the public API I suspect would give a misleading signal to typical JuMP users that something more dramatic has changed.

3 Likes
10

We discussed this on today’s developer call. The plan is:

Would there be any real downside of just making this a 2.0?

Our development style has been to move slowly and carefully. It took 10 years to get to JuMP 1.0, so we won’t be making a 2.0 release anytime soon. Note that no part of the public API will break; this is exclusively a refactoring of the internals.

In the MPSGE case, you don’t support JuMP 1.0 yet anyway, so you can probably just skip from 0.22 to v1.2.

6 Likes
11

To close the loop on this: JuMP v1.2.0 has been released!

Please open an issue or post below if you encounter any issues updating from JuMP v1.1 to v1.2.

5 Likes