Exciting! 
@franckgaga thereās a decent change that Uno with the SQP settings would beat Ipopt for MPC problems
3 Likes
Iām in a good position to know that a free and positive publicity is always appreciated for open source projects. I finished the implementation of exact Hessian support in ModelPredictiveControl.jl. I just tested UnoSolver.jl with filtersqp, a multiple shooting transcription, and exact Hessian computed by sparse ForwardDiff.jl on the inverted pendulum case study (unstable system + highly nonlinear/nonconvex). Itās about 2.1 times faster than Ipopt.jl, all other things being equal. Excellent work @cvanaret and @amontoison ! 
The L-BFGS will be a nice addition in the case where Hessian is too expensive to compute. Even in the small-in-size case study above (in which sparse ForwardDiff.jl is typically well-suited), a fair amount of time is spent in the Hessian evaluation.
13 Likes
Thatās awesome news @franckgaga! 
Thanks for the feedback.
The underlying QP solver (BQPD) accepts Hessian-vector products, so if you think that theyāre cheaper than forming the explicit Hessian for your application, feel free to give it a go.
Also, Iām curious to see how the (undocumented) funnelsqp preset performs. It replaces the filter method with a funnel method, which allows more non-monotonicity. Let me know!
4 Likes
It is not supported by the MOI.VectorNonlinearOracle for now, but maybe in the future. Still, itās already a huge efficiency improvement compared to the old ways of doing this with @operator (computing individual Hessians for each components of the nonlinear constrain vectors and manually evaluating the weighted sum, while using splatting when there are multiple decision variables).
I just tested it on the same case study and itās very similar in performance compared to filtersqp.
2 Likes
We have just submitted the revision of the Uno paper to Math Programming Computation 
We introduce a unifying framework for Lagrange-Newton methods (SQP and barrier) with 8 ingredients (a major update since the first submission in June 2024), and describe state-of-the-art strategies through the prism of the unifying framework. We then present the basic algorithmic design of Uno and show how various nonlinear optimization methods fit within the architecture. Finally, we provide preliminary numerical results and compare Uno against state-of-the-art solvers.
Happy to get some feedback and suggestions!
10 Likes
I just tried Uno with the filtersqp present as the inner optimizer for a mixed-integer MPC problem, it was 6x faster despite using tighter tolerances than Ipopt. What a nice addition to the julia optimizer ecosystem 
13 Likes
Wow thatās super cool! Thanks for your positive feedback
@baggepinnen @franckgaga would you mind sharing some info about your instances (#variables, #constraints, #inequality constraints, #sparsity, etc)?
@franckgaga not sure thatās relevant but I think your benchmarks have changed following our recent discussions on sparse autodiff?
It would be nice to show some results for larger-scale problems, with thousands or even millions of parameters, in which dense Hessians are not practical.
A nice collection of high-dimensional non-convex optimization test problems in physics can be found at invrs-gym, for example.
3 Likes
Itās not a large-scale problem. As a side note, I personally think that large-scale problems are ill-suited for online real-time optimization like MPC. Surrogate or reduced-order plant models should be used if possible. That being said, hereās the problem sizes:
- 62 decisions variables (2 for the manipulated input, 60 for the states over the prediction horizon)
- 60 nonlinear equality constraints (the state defects over the prediction horizon)
- 40 linear inequality constraints (operational limits for the manipulated input)
- 403 non-zeros in the Hessian of the Lagrangian
- 271 non-zeros in the nonlinear equality constraint Jacobian
Yes, my recent modifications to the coloring parameters may have improved the performances, but Iām no longer able to fairly compare the performances of UnoSolver.jl vs Ipopt.jl right now, since a recent improvement in the linear constraint handling was introduced in the MOI of Ipopt.jl, but not in UnoSolver.jl (details here).
1 Like
My problem wasnāt very large either, 124 variables, 100 constraints (100 equality, 0 inequality), 400 nonzeros in the Hessian of the Lagrangian and 580 in the constraint Jacobian.
I am not using JuMP, but I am using Uno through the MOI interface wrapped by Optimizaiton.jl. I believe that this interface supports vector-Jacobian products, but Iām not sure if they are used? It does not look like Optimization.jl supports Lag-Hess-vector products, but maybe it could be made to? @SebastianM-C
Objective evaluations: 2
Constraints evaluations: 2
Objective gradient evaluations: 2
Jacobian evaluations: 2
Hessian evaluations: 1
Number of subproblems solved: 1
More or less identically to the filtersqp present for the particular problem Iām testing on.
Coming up with a good surrogate model is hard, and so is coming up with good terminal ingredients. The larger problems we can solve efficiently, the less those two difficult things are needed, moving the effort from the user to the solver
2 Likes
As I understand we would need to add Lagrangian Hessian vector products in OptimizationBase and then the optimization backends that support that will be able to opt in.
As for vector-Jacobian products, we have both cons_vjp and cons_jvp in OptimizationBase, but only OptimizationMadNLP is using that at the moment.
2 Likes
We had major issues with the GC in Unoās NLPModels interface. This is now fixed in UnoSolver.jl v0.1.6 
5 Likes