# ANN: upcoming refactoring of JuMP's nonlinear API
**URL:** https://discourse.julialang.org/t/ann-upcoming-refactoring-of-jumps-nonlinear-api/83052
**Category:** Optimization (Mathematical)
**Tags:** jump
**Created:** [June 20, 2022, 7:25am UTC](https://discourse.julialang.org/t/ann-upcoming-refactoring-of-jumps-nonlinear-api/83052 "2022-06-20T07:25:06Z")
**Posts on this page:** 11
**Page:** 1
### Author:  [@odow](https://discourse.julialang.org/u/odow)
#### Post date: [June 20, 2022, 7:25am UTC](https://discourse.julialang.org/t/ann-upcoming-refactoring-of-jumps-nonlinear-api/83052/1 "2022-06-20T07:25:07Z")
**tl;dr:**
The next feature release of JuMP (probably v1.2.0) introduces a large
refactoring of JuMP’s nonlinear API. If you have code that accessed private
features of the nonlinear API, such as `JuMP._Derivatives` or `model.nlp_data`,
your code will break. If you used only the public, documented API such as
`register`, `@NLconstraint` and `num_nonlinear_constraints`, this change does
not affect you. To try the uncoming release, use
`import Pkg; Pkg.pkg"add JuMP#od/moi-nonlinear"`, then restart Julia for the change to take effect.
The relevant pull request is [https://github.com/jump-dev/JuMP.jl/pull/2955](https://github.com/jump-dev/JuMP.jl/pull/2955)
## What are we doing?
Over the last few months, we have been refactoring how JuMP supports nonlinear
programs. This involved moving and re-organizing a large amount of code from
JuMP into MathOptInterface.
The result is the new `MOI.Nonlinear` submodule in MathOptInterface, with a
documented public API for creating and dealing with nonlinear programs. Read
more about it here: [Overview · MathOptInterface](https://jump.dev/MathOptInterface.jl/stable/submodules/Nonlinear/overview/)
However, as part of this work we are removing code from JuMP. This code was
internal, undocumented, and not intended for public use. Most of it was
contained in the `JuMP._Derivatives` submodule, but we also made changes such
as removing `model.nlp_data`.
## Why did we do this?
The nonlinear code in JuMP was a clear example of technical debt. It was
complicated, convoluted, and largely undocumented. People wanting to extend JuMP
for nonlinear programs were forced to use a range of hacks that relied on
undocumented internals.
The new `MOI.Nonlinear` submodule offers a stable, documented, and public API
for people to build JuMP extensions on. It also enables new features like
swappable automatic differentiation backends, and hessians of user-defined
functions.
We originally considered that any change in the nonlinear API would be a
breaking v2.0.0 release of JuMP and occur at least two years after the release
of JuMP 1.0. However, we implemented the changes quicker than expected, and we were
able to do so in a way that does not break the public nonlinear API. Therefore,
we elected to classify this as a non-breaking feature release.
## Does it affect me?
If you have any code that called private features of the JuMP nonlinear API,
such as `JuMP._Derivatives` or `model.nlp_data`, your code will break.
If you used only the public, documented API such as `register`, `@NLconstraint`
and `num_nonlinear_constraints`, this does not affect you.
## What are the next steps
Try the uncoming release as follows:
`import Pkg; Pkg.pkg"add JuMP#od/moi-nonlinear"`.
If you find any bugs or changes in performance, please post below, or open a
GitHub issue. Once we’re happy that there are no issues with the changes, we
will merge the pull request, and then release a new version of JuMP with the changes.
---
### Author:  [@odow](https://discourse.julialang.org/u/odow)
#### Post date: [June 20, 2022, 7:27am UTC](https://discourse.julialang.org/t/ann-upcoming-refactoring-of-jumps-nonlinear-api/83052/2 "2022-06-20T07:27:23Z")
---
### Author:  [@baggepinnen](https://discourse.julialang.org/u/baggepinnen)
#### Post date: [June 20, 2022, 7:31am UTC](https://discourse.julialang.org/t/ann-upcoming-refactoring-of-jumps-nonlinear-api/83052/3 "2022-06-20T07:31:30Z")
Very cool, thanks for your work on this! 🙂
I’m looking at [Overview · MathOptInterface](https://jump.dev/MathOptInterface.jl/stable/submodules/Nonlinear/overview/#Multivariate-operators) and [API Reference · MathOptInterface](https://jump.dev/MathOptInterface.jl/stable/submodules/Nonlinear/reference/#MathOptInterface.Nonlinear.register_operator), it looks like the function is taking multiple arguments `x...`. Does user-defined functions from arrays to arrays also work in this new interface?
---
### Author:  [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)
#### Post date: [June 20, 2022, 8:52am UTC](https://discourse.julialang.org/t/ann-upcoming-refactoring-of-jumps-nonlinear-api/83052/4 "2022-06-20T08:52:06Z")
Awesome work!
> [@odow](#):
>
> The new `MOI.Nonlinear` submodule offers a stable, documented, and public API
> for people to build JuMP extensions on. It also enables new features like
> swappable automatic differentiation backends, and hessians of user-defined
> functions.
How does that correspond to:
> - **Goal: limit the scope to improve robustness** Most other AD packages accept arbitrary Julia functions as input and then trace an expression graph using operator overloading. This means they must deal (or detect and ignore) with control flow, I/O, and other vagaries of Julia. In contrast, `ReverseAD` only accepts functions in the form of [`Nonlinear.Expression`](https://jump.dev/MathOptInterface.jl/stable/submodules/Nonlinear/reference/#MathOptInterface.Nonlinear.Expression), which greatly limits the range of syntax that it must deal with. By reducing the scope of what we accept as input to functions relevant for mathematical optimization, we can provide a simpler implementation with various performance optimizations.
So you can swap AD backends, but it still needs to be something that only uses `register`ed functions in `Nonlinear.Expression`s, where this AD backend changes the way the derivative is created for the `register`ed function?
---
### Author:  [@odow](https://discourse.julialang.org/u/odow)
#### Post date: [June 20, 2022, 5:39pm UTC](https://discourse.julialang.org/t/ann-upcoming-refactoring-of-jumps-nonlinear-api/83052/5 "2022-06-20T17:39:44Z")
> [@baggepinnen](#):
>
> Does user-defined functions from arrays to arrays also work in this new interface?
No. Nothing changes from the current JuMP interface, which requires splatting.
---
### Author:  [@odow](https://discourse.julialang.org/u/odow)
#### Post date: [June 20, 2022, 5:45pm UTC](https://discourse.julialang.org/t/ann-upcoming-refactoring-of-jumps-nonlinear-api/83052/6 "2022-06-20T17:45:24Z")
> [@ChrisRackauckas](#):
>
> So you can swap AD backends, but it still needs to be something
Each AD gets given the full expression graph, along with callbacks for how to numerically evaluate the function, gradient, a hessian of each operator. Then they have to produce a `MOI.AbstractNLPEvaluator` which implements the MOI callbacks like `eval_constraint_jacobian` and `MOI.hessian_lagrangian_structure`.
We have a few implementations already which gives us confidence this is a reasonable thing to do:
- The `SparseReverseAD` in JuMP
- `SymbolicAD` in [GitHub - odow/MathOptSymbolicAD.jl](https://github.com/odow/SymbolicAD.jl), which uses Symbolics.jl to compute sparse derivatives, but which uses some tricks to avoid computing the symbolic derivative of the full problem
- `MadDiff`: [GitHub - sshin23/MadDiff.jl: An automatic differentiation and algebraic modeling package](https://github.com/sshin23/MadDiff.jl)
For @ccoffrin’s [AC-OPF problems](https://discourse.julialang.org/t/ac-optimal-power-flow-in-various-nonlinear-optimization-frameworks/78486), SymbolicAD is 3-5x faster than `SparseReverseAD`, but on other problems it can be much worse.
---
### Author:  [@davidanthoff](https://discourse.julialang.org/u/davidanthoff)
#### Post date: [June 23, 2022, 2:29pm UTC](https://discourse.julialang.org/t/ann-upcoming-refactoring-of-jumps-nonlinear-api/83052/7 "2022-06-23T14:29:17Z")
I understand that technical making this a minor release is correct, but on the other hand my sense is that there are packages out there that depend on JuMP and that rely on internals that will break if this gets released as 1.2, right? Complementarity.jl is one that comes to mind, not sure whether there are others as well?
In my mind, in such a situation it would be more helpful for users to just make this a 2.0 release. If I’m a user of a package that depends on JuMP internals, I a) might not even be aware of that, and b) can’t really do anything about it. In the end the user experience will just be that things don’t work, and I think it would be nicer to try to prevent that.
Plus, I’m not aware how a depending package like Complementarity.jl could even signal that it doesn’t work with JuMP 1.2, but it does work with 1.1? Maybe I’m missing something there…
Would there be any real downside of just making this a 2.0?
And also: awesome work, very excited to try it!
---
### Author:  [@miles.lubin](https://discourse.julialang.org/u/miles.lubin)
#### Post date: [June 23, 2022, 5:56pm UTC](https://discourse.julialang.org/t/ann-upcoming-refactoring-of-jumps-nonlinear-api/83052/8 "2022-06-23T17:56:11Z")
> [@davidanthoff](#):
>
> Plus, I’m not aware how a depending package like Complementarity.jl could even signal that it doesn’t work with JuMP 1.2, but it does work with 1.1? Maybe I’m missing something there…
Packages that depend on internal JuMP APIs should indicate compatibility only with patch releases. I believe the [tilde specifier](https://pkgdocs.julialang.org/v1/compatibility/#Tilde-specifiers) should work. It’s important that we make sure any potentially broken packages have accurate compatibility specs.
---
### Author:  [@ccoffrin](https://discourse.julialang.org/u/ccoffrin)
#### Post date: [June 23, 2022, 6:07pm UTC](https://discourse.julialang.org/t/ann-upcoming-refactoring-of-jumps-nonlinear-api/83052/9 "2022-06-23T18:07:04Z")
> [@davidanthoff](#):
>
> Would there be any real downside of just making this a 2.0?
I second Miles’ remarks and would add, given that JuMP has been extremely stable since v0.18 with only minor breaking changes in the last several years leading up to v1.0. Updating to v2.0 with no breaking changes to the public API I suspect would give a misleading signal to typical JuMP users that something more dramatic has changed.
---
### Author:  [@odow](https://discourse.julialang.org/u/odow)
#### Post date: [June 23, 2022, 9:35pm UTC](https://discourse.julialang.org/t/ann-upcoming-refactoring-of-jumps-nonlinear-api/83052/10 "2022-06-23T21:35:47Z")
We discussed this on today’s developer call. The plan is:
- Wait at least another week for people to become aware
- Fix any compat bounds within packages that depend on the breaking internals. In the complementarity case, it doesn’t (yet) support JuMP 1.0 anyway: [Add JuMP 1.0 to compat by davidanthoff · Pull Request #64 · chkwon/Complementarity.jl · GitHub](https://github.com/chkwon/Complementarity.jl/pull/64)
- Retroactively fix compat bounds in the General registry to avoid issues around installing older versions of packages without the correct compat bound.
> Would there be any real downside of just making this a 2.0?
Our development style has been to move slowly and carefully. It took 10 years to get to JuMP 1.0, so we won’t be making a 2.0 release anytime soon. Note that no part of the public API will break; this is exclusively a refactoring of the internals.
In the MPSGE case, you don’t support JuMP 1.0 yet anyway, so you can probably just skip from 0.22 to v1.2.
---
### Author:  [@odow](https://discourse.julialang.org/u/odow)
#### Post date: [August 16, 2022, 8:24am UTC](https://discourse.julialang.org/t/ann-upcoming-refactoring-of-jumps-nonlinear-api/83052/11 "2022-08-16T08:24:23Z")
To close the loop on this: JuMP v1.2.0 has been released!
Please open an issue or post below if you encounter any issues updating from JuMP v1.1 to v1.2.