Hierarchical/compositional construction of large optimization models

Specific Domains Optimization (Mathematical)
2

For other readers, here’s the link to the JuMP docs, which summarizes my suggested structure:

Design patterns for larger models · JuMP?

That list is fairly comprehensive. The NREL/SIIP PowerSystems.jl is probably the largest example with the most engineering time put into it. They went full modularity, and full performance (sometimes at the expense of some complicated code to pre-allocate JuMP containers).

The power-focused applications are also somewhat niche, because they have some unique features that aren’t in other applications (like a transmission graph, or multiple time structures, and very flexible generator combinations with different technologies all producing the same commodity).

I think the most important thing to start with is an understanding of the data. Ultimately, the JuMP optimization part will be a very small component of the overall application.

  • What at the the inputs and outputs at each layer of the hierarchy?
  • What format should the inputs and outputs be?
  • How can you setup, test, and validate each layer of the hierarchy independently?

I’d also advise that unless necessary, you avoid fancy tricks. Do the simplest thing possible, even if it is slightly slower than a more clever thing.

As a cautionary tale of something that works, but which I don’t think we got right (and since I’m somewhat responsible), take a look at GitHub - EPOC-NZ/JADE.jl.

Despite all the mess, the actual SDDP model is 600 lines, but surprisingly clean and easy to read and understand: JADE.jl/src/model.jl at f2e753eac0f6d7354c3b412da830c0d4c1bcec9c · EPOC-NZ/JADE.jl · GitHub

The backstory is the Julia code was a 2016 re-implementation of a C++ application from 2011, which was a re-implementation of a series of AMPL scripts from 2008. And then between 2016 and now, a bunch of things got added to it in various places. (And during that time, Julia and JuMP and SDDP.jl all changed quite a lot.). So at no point did we ever step back and design a nice representation of the data layer.

I guess my point is ignore the JuMP part to start with. If you have good data representation, the JuMP part will be clean and simple. If you have a nice JuMP model and messy data, the entire thing will be a mess.

1 Like