Lately I have been switching back between CVXPY and JuMP.jl a lot (mainly for work).
One of the many nice features CVXPY has, is that it can automatically reformulate problems which are DCP-compliant through reductions. DCP is also used to verify convexity of a program. DCP itself can also be interesting and useful outside of optimization. For example, in machine learning, control, and as a general tool in the symbolics toolbox.
Nope. My current goal with Convex is to keep it minimally maintained.
I know you already know this, but just pointing out to other readers that Convex.jl is mainly intended to be used standalone by itself without JuMP. It is a CVXPY equivalent. (It just carries some technical debt from design choices made when Julia was v0.2 (!!!))
The general DCP algorithm that CVXPY uses is fairly inefficient, not in a code sense but in a theoretical sense (though Iâm sure the code has things it could improve on too), and I have some ideas for improvements to the algorithm that would be good for a masterâs / PhD student project, but just no one to work on it right now. It already is a small generalization beyond standard DCP ([2407.05261] Disciplined Geodesically Convex Programming), but it still has a long ways to go.
Could you elaborate on what that means? I am aware of the limitations of DCP, if thatâs what youâre referring to. Not everything that is convex, is DCP. But usually small reformulations can be made to pass DCP checks anyway.
This is hard to follow for those of us who are not mathematicians. Is there any intention of extending this with reductions so we can reformulate JuMP problems efficiently? Considering Oscarâs answer, the path DCP + JuMP would otherwise be dead. Supporting something like dcp2cone would already open up a ton of applications.
p.s. I understand all of this is an enormous amount of work. I am just curious whether people are thinking about this and if we ever want to match what the CVXPY ecosystem offers.
Convex.jl implicitly does this for the most part, and you could likely make use it by doing API ¡ Convex.jl (or if you donât want a file, copy the implementation and skip the last line: Convex.jl/src/problems.jl at ba3a8f2047930208c0d72dfba9e292d27fc745b4 ¡ jump-dev/Convex.jl ¡ GitHub). However if you want to restrict the supported cones etc further than what convex currently emits youâd need to modify that code to give it an MOI optimizer with those restrictions. The MOI system is very general and you can be pretty picky about how you want to formulate it but it requires more coding than the simple cvxpy API. That being said if there are certain restrictions that are commonly desired one would only need to write the code once and PR it to Convex.jl to add the function. I donât think there is any design issues in Convex preventing it, on the contrary I think our use of MOI means that kind of task is actually quite easy.
However I may be missing something in terms of the integration you want with JuMP.
Iâm curious about precisely which kind of problems you want to reformulate with DCP. As far as I know the motivation for introducing it was to automatically reformulate exotic cones in terms of LPs/SDPs. But since DCP was introduced conic solvers have become able of solving several exotic cones directly, which is much more efficient than reformulating. For example, both MOSEK and SCS can now handle the power cone and the exponential cone directly, whereas Hypatia supports a long list of exotic cones.
The MOSEK cookbook has quite an extensive list of reformulations, also for problems involving exponential cones. If you know all of these formulations by heart and youâre okay doing the math yourself then probably you donât really need DCP. It gets more complicated when your solver doesnât support exponential cones directly and you have to resort to SOC approximations. You would present these reformulations as atoms to the user, while the backend decides on the best reduction based on the solver youâre using.
Iâll just say that the whole story of e-graphs, tearing algorithms, mixing analytical solutions, and all of the other symbolic-numerics research going on intersects really nicely with DCP, and thus many of the observations weâve had in those other areas are also clear improvements to the algorithms there.
Perhaps Iâm missing something, but SymbolicAnalysis.jl doesnt have MOI or any conic solver listed as a dependency?
Anyway, the CVX folks have been discussing something along these lines
Iâd encourage anyone interested to just try something out and experiment. The design space is pretty large. You donât need to incrementally build on what people have done before.
Yeah it uses Manifolds.jl and ManOpt.jl via Optimization.jl (OptimizationManOpt.jl) since in theory it could be used for targeting more than just convex optimization and instead does geodesic convexity. But the biggest use case should be just to convex so it should target MOI. The code just needs work.
There are a few things Iâd aim for if I was starting from scratch.
The first two are essentially âConvex.jl but written with 12 years experience in how to write good Julia packagesâ:
First-class support for Base.Array and broadcasting. Currently Convex keeps everything as AbstractExpr objects that remember their size. Except in rare cases, these objects do not support Julia broadcasting. So you write exp(Variable(2))::Convex.AbstractExpr instead of x = Variable(2)::Vector{Convex.Variable} and exp.(x)::Vector{Convex.AbstractExpr}
Great debugging information to explain how Convex transformed the problem from the user to the conic solver. JuMP has JuMP.print_active_bridges(model), so this would be a layer above that that showed how the nonlinear expressions were lowered to conic constraints.
The stretch goal that Chris and the CVX folks are aiming at is really:
Can you provide a rule system for the user to lower nonlinear expressions to known DCP forms. One example is sqrt(sum_squares(x)) <= 1 is not DCP, even though norm(x) <= 1 is. I want to be able to say something like sqrt(sum_squares(_x)) => norm(_x). I donât know what the right approach to this is.
Thatâs a pretty good list. Iâd add also not just transforming something to DCP, but also the problem of finding the minimal DCP representation. Also specializations on array operations / linear algebra to preserve structure.
I wonder if this could be used as common middleware layer between cvxpy/jump/? and a DCP representation. Everyone involved seems to be talking about the same things and has the same goals.
*just realized CvxLean was also referenced in the CVX forum post
There will somewhen the next few weeks more support for Manopt.jl (no capital o) directly from within JuMP.jl.
For the support within Optimization.jl, I did not yet find the time/motivation to dive deep enough into Optimization.jl to help much, the current support is very very minimalistic.
I would carefully phrase that as: That paper could also have been written a bit more accessible on what geodesic convexity implies and where that is useful.
That would certainly help. But it also depends on what the intended audience of the paper and the SymbolicAnalysis.jl package is. It doesnât seem to be people like me who are mostly users of convex optimization tools. And Iâm not saying thatâs a bad thing, but my goal is to deploy to production which means I need the conic form so generic solvers can understand it.
