So there are a bunch of observations so far which I want to get some insights into. I’m describing issues with nonlinear solver btw.
As soon as I’m using atan with 2 arguments JuMP seems registers this function for me (it even produces corresponding warning). After that no Hessian available (which is an expected but unfortunate behaviour of JuMP). The same if I manually registered atan2 function and used it in constraints or objective.
Also for the same problems expressed in JuMP and CasADi I usualy have same number of nonzero elements in Jacobian but JuMP produces Hessian matrices with order of magnitude more nonzero elements than CasADi. Consequently on complex problems CasADi works several times faster then JuMP (using the same IpOpt NLP solver). I’m using CasADi.jl which in turns runs Python CasADi library.
What are the ways to have atran2 and automatic Hessian in JuMP or more generally any user registerd function with automatic Hessian?
Does anyone knows why CasADi’s Hessian matrices are order of magnitude sparsier then JuMP’s?
The “consequently” doesn’t follow. There are many operations that take place in computing hessians and processing them inside Ipopt. Handling nonzeros is one part of that. JuMP tends to produce multiple nonzero elements corresponding to the same entry in the matrix if there are multiple algebraic terms contributing to that element. Ipopt documents that, “The individual elements of the matrix can be listed in any order, and if there are multiple items for the same nonzero position, the values provided for those positions are added.” It’s possible that doing this preprocessing inside JuMP could be faster than in Ipopt, we never ran that experiment.
Of course this discussion doesn’t help you solve the problem faster, but my point is that more detailed profiling would be needed to conclude why JuMP is slower than CasADi in this case.
Univariate user-defined functions are compatible with hessian computation. Do you need the two-argument version of atan? Are both arguments related to decision variables?
Thank you @miles.lubin for detailed explanation on hessians processing in ipopt. But still on 3 different tasks of different sizes from hundreds to thousands of decision variables and approximately the same number of nonlinear equality constraints casadi significantly outperforms JuMP in my tasks. I converted to Julia from python where I used to work with casadi but still have to use python’s casadi from julia as well)
As for atan2 unfortunately both arguments are decision variables for one of my tasks.
Also I have my cost function and nonlinear equality constraints have nontrivial algorithms inside. So I can’t code them as expressions into JuMP macros by hand. I have to pass Symbolics.jl variables to my functions first in order to extract expressions. Then I have to use MacroTools to convert these expressions to something JuMP would accept substituting :x with $x and so on…
With casadi I just pass its variables to my functions and it works. So casadi still not only faster during optimization but also much more convenient to use during task description phase.
But I still like julia expressiveness, powerful typesystem and its composability of libraries through multiple dispatch. And hope it will payoff eventually.
Understood. The nonlinear functionality in JuMP was designed to match or exceed the functionality of AMPL, GAMS, Pyomo, and similar systems. We don’t claim to be a substitute for CasADi; I apologize if you got this impression and would be curious where it came from, so we can avoid sending users down the wrong path. If you’re happy with CasADi, then by all means keep using it.
Well my tasks are mostly on trajectory optimization, optimal control which you have a couple of examples in your Nonlinear program tutorial. I hope one day you might consider CasADi to match and exceed as well. Because you are already almost there.
There is might be some performance penalties because of how you working with hessians (or might be not).
One more inconvenience from my point of view is inability to use decision variables in nonlinear expressions directly. I cant pass them to my function to extract expression as a return as long as my function contains any nonlinear and non-quadratic calculations. One have to use macros to code nonlinear expressions which is not convenient for large problems with hundreds of constraints. Or have to generate expressions programmatically extracting them first with Symbolics.jl from function… I don’t know if there is a simpler way.
