Integrating MathOptInterface into an industry-scale project (e.g. constraint management)

Specific Domains Optimization (Mathematical)
1

Hi,
I have been tasked by my employer to think about a general “Operations Research Framework” that guides future operations research projects, both at the level of project management (how to organise this in an agile way, processes, etc.) but also at the level of best practices for the codebase. The types of optimization problem that this should cover are quite general (sometimes simple LPs but also sometimes highly nonlinear problems with black-box character for which anytime algos or metaheuristics would be the right choice) and MOI seems like the right level of abstraction to formulate how models should be written down, constraints documented and annotated, etc.

Part of the requirement will be that it should very simple to present one’s current understanding of the optimization problem to stakeholders, in a language that they understand and including giving a full list of all constraints that are in place. For this reason I’m thinking about how to integrate MOI into some kind of constraint management system. For instance, there could be Constraint structs with human-readable metadata and descriptions but also fields that correspond to the MOI ConstraintIndex and then these could be stored an SQL database or something like this. It would be great if this could be done without dislinking the “stakeholder-facing” version of the problem from the “solver-facing” version, in order to avoid the two versions moving out of sync, etc.

I find it hard to get started on this, moving the model definition away from the simple enumeration of variables-constraints-objective in a single file, and am wondering about how other people manage the OR problems in practice, whether tools for this exist, etc. Any help on this is much appreciated.

Cheers, Paul

2

MOI seems like the right level of abstraction to formulate

No. You should use JuMP instead. MOI is too complicated, and lacks good support for nonlinear problems.

Part of the requirement will be that it should very simple to present one’s current understanding of the optimization problem to stakeholders, in a language that they understand and including giving a full list of all constraints that are in place.

Use JuMP. The algebraic model can be simple to present the understanding of the optimization model to stakeholders.

moving the model definition away from the simple enumeration of variables-constraints-objective in a single file

Many projects have success in a functional form with method dispatch to create different sets of variables or constraints.

PowerSimulations is probably the largest example of this:

A concrete example might look like:

struct Object
    weight::Float64 
    profit::Float64
end

struct KnapsackData
    objects::Vector{Object}   
    capacity::Float64 
end

abstract type AbstractKnapsack end

struct BinaryKnapsack <: AbstractKnapsack end
struct IntegerKnapsack <: AbstractKnapsack end

function add_knapsack_variables(model, data::KnapsackData, ::BinaryKnapsack)
    @variable(model, x[1:length(data.objects)],  Bin)
    return
end

function add_knapsack_variables(model, data::KnapsackData, ::IntegerKnapsack)
    @variable(model, x[1:length(data.objects)] >= 0,  Int)
    return
end

function add_capacity_constraint(model, data::KnapsackData, ::AbstractKnapsack)
    x = model[:x]
    N = length(x)
    @constraint(
        model, 
        capacity, 
        sum(data.objects[i].weight * x[i] for i in 1:N) <= data.capacity,
    )
    return
end

function add_profit_objective(model, data::KnapsackData, ::AbstractKnapsack)
    x = model[:x]
    N = length(x)
    @objective(model, Max, sum(data.objects[i].profit * x[i] for i in 1:N))
    return
end

function knapsack_model(data::KnapsackData, config::AbstractKnapsack)
    model = Model()    
    add_knapsack_variables(model, data, config)
    add_capacity_constraint(model, data, config)
    add_profit_objective(model, data, config)
    return model
end

data = KnapsackData([Object(rand(), rand()) for i in 1:20], 10.0)
binary_model = knapsack_model(data, BinaryKnapsack())
integer_model = knapsack_model(data, IntegerKnapsack())

A key benefit of using JuMP is that you can write the algebraic model in a readable form, and then combine it with the power of Julia to build a framework that suits you. A downside of the generality is that there aren’t off-the-shelf solutions.

8 Likes
3

I should actually turn this into a tutorial. It’s a useful approach to constructing larger problems.

7 Likes
4

Right, there could be a blog post about “patterns of composition in optimization modeling”!

1 Like
5

I would contribute to writing that!

2 Likes
6

Thanks @odow, that was very helpful.

MOI is too complicated, and lacks good support for nonlinear problems.

Interesting. I thought it was the other way around: Since the models you can formulate in MOI is a strict superset of those that you formulate in JuMP, the solvers that support MOI would be a strict superset of those that support JuMP, but I must be mistaken.

If I can be of help writing such a tutorial or blog post, I’m more than happy to contribute. Of course, as you can see from the fact that I asked the question, I don’t bring experience but see that the approach is basically straightforward.

2 Likes
7

JuMP is in some sense “mathoptinterface with nicer syntax.”

There is no difference between the supported solvers or the problem types. In fact, JuMP makes it easier to specify nonlinear problems; the nonlinear interface at the moi level is difficult to implement.

8

I’ll make a start with the tutorial this morning. Once I have a draft I’ll post it here and people can comment with feedback.

3 Likes
9

Comments appreciated: https://github.com/jump-dev/JuMP.jl/pull/2812

1 Like
10

On the question relating to broader structuring and presentation of models, I thought that there were a number of nice examples in this year’s JuMP-dev conference

I recall an interesting example described in AnyMOD.jl: A Julia package for creating energy system models - Leonard Göke of using DataFrames to manage constraints.

Also very worthwhile: A Brief Introduction to InfrastructureModels - Carleton Coffrin

And to back up @odow’s recommendation of PowerSimulations, see this talk from JuliaCon:
[https://live.juliacon.org/talk/88EDGD]

2 Likes